vcstools modules

analyse_psf

Scripts to analyise the output PSFs

vcstools.analyse_psf.plot_imaging_psf(fits_file, output_name='imaging_psf.png', normalise=False, vmin=None, centre_size=None, fft_abs=False, fft_angle=False, centre=False)

Plots the imaging PSF (from WSCLEAN for example).

Parameters

fits_filestr

The PSF fits file gained from imaging (WSCLEAN).

output_namestr, optional

Output plot name.
Default: imaging_psf.png.

normaliseboolean, optional

Normalise the PSF.
Default: False.

vminfloat, optional

Minimum value of plot.
Default: None.

centre_sizeint, optional

The radius in pixels to plot from the centre.
Default: None.

fft_absboolean, optional

Take the fft of the PSF and plot the amplitude.
Default: False.

fft_angleboolean, optional

Take the fft of the PSF and plot the phase.
Default: False.

vcstools.analyse_psf.plot_pabeam(dat_file, output_name='pabeam_psf.png')

Makes a polar plot over the sky response with the data file output from pabeam.py

Parameters

dat_filestr

The data file output from pabeam.py.

output_namestr, optional

Output plot name.
Default: pabeam_psf.png.

vcstools.analyse_psf.plot_pabeam_ra_dec(dat_file, output_name='pabeam_psf.png', normalise=False, vmin=None)

Plots the PSF data file output pabeam.py when using the –ra_dec_projection option.

Parameters

dat_filestr

The PSF data file output pabeam.py when using the –ra_dec_projection option

output_namestr, optional

Output plot name.
Default: pabeam_psf.png.

normaliseboolean, optional

Normalise the PSF.
Default: False.

vminfloat, optional

Minimum value of plot.
Default: None.

vcstools.analyse_psf.plot_psf_comparison(imaging_psf, pabeam_psf, c_pabeam_psf)

Compare the PSFs FWHM along RA and declination from imaging, pabeam.py and vcsbeam’s mwa_tied_array_beam_psf. Outputs a plot called comparing_psf_cuts.png.

Parameters

imaging_psfstr

The PSF fits file gained from imaging (WSCLEAN).

pabeam_psfstr

The PSF data file output from pabeam.py.

c_pabeam_psfstr

The PSF data file output from vcsbeam’s mwa_tied_array_beam_psf.

vcstools.analyse_psf.plot_track_beam_response(response_file, output_name='vcsbeam_response.png', time_max=-1)

Plot the data file output from vcsbeam’s mwa_track_primary_beam_response

Parameters

response_filestr

The PSF data file output from vcsbeam’s mwa_track_primary_beam_response.

output_namestr, optional

Output plot name.
Default: vcsbeam_psf.png.

time_maxint, optional

Maximum number of seconds to process.
Default: -1.

vcstools.analyse_psf.plot_vcsbeam_psf(psf_file, output_name='vcsbeam_psf.png', normalise=False, vmin=None)

Plot the PSF data file output from vcsbeam’s mwa_tied_array_beam_psf

Parameters

psf_filestr

The PSF data file output from vcsbeam’s mwa_tied_array_beam_psf.

output_namestr, optional

Output plot name.
Default: vcsbeam_psf.png.

normaliseboolean, optional

Normalise the PSF.
Default: False.

vminfloat, optional

Minimum value of plot.
Default: None.

vcstools.analyse_psf.read_pabeam_ra_dec(dat_file)

Read in the PSF data file output from pabeam.py when using the –ra_dec_projection option.

Parameters

dat_filestr

The PSF data file output from pabeam.py

Returns

ranumpy.array, (Nx, Ny)

The RA (degrees) in the meshgrid format

decnumpy.array, (Nx, Ny)

The declination (degrees) in the meshgrid format

powernumpy.array, (Nx, Ny)

The the data values of the fits file in the meshgrid format

vcstools.analyse_psf.read_psf_fits(fits_file, centre_size=None)

Read in an imaging PSF fits file (from WSCLEAN for example).

Parameters

fits_filestr

The PSF fits file gained from imaging (WSCLEAN).

centre_sizeint, optional

The radius in pixels to plot from the centre.
Default: None.

Returns

ravnumpy.array, (Nx, Ny)

The RA (degrees) in the meshgrid format.

decvnumpy.array, (Nx, Ny)

The declination (degrees) in the meshgrid format.

fits_datanumpy.array, (Nx, Ny)

The the data values of the fits file in the meshgrid format.

vcstools.analyse_psf.read_vcsbeam_psf(psf_file)

Read in the PSF data file output from vcsbeam’s mwa_tied_array_beam_psf

Parameters

psf_filestr

The PSF data file output from vcsbeam’s mwa_tied_array_beam_psf.

Returns

ranumpy.array, (Ny, Nx)

The RA (degrees) in the meshgrid format.

decnumpy.array, (Ny, Nx)

The declination (degrees) in the meshgrid format.

powernumpy.array, (Ny, Nx)

The the data values of the fits file in the meshgrid format.

beam_calc

Functions used to calculate or simulate the MWA tile beam.

vcstools.beam_calc.beam_enter_exit(powers, duration, dt=296, min_z_power=0.3)

Calculates when the source enters and exits the beam

Parameters

powerslist, (ntimes, nfreqs)

Powers for the duration every dt and freq.

durationint

Duration of the observation according to the metadata in seconds.

dtint, optional

The time interval of how often powers are calculated. Default: 296.

min_z_powerfloat, optional

Zenith normalised power cut off.
Default: 0.3.

Returns

dect_beg_norm, dect_end_normfloat

Fraction of the observation when the source enters and exits the beam respectively.

vcstools.beam_calc.field_of_view(obsid, common_metadata=None, dur=None)

Will find the field-of-view of the observation (including the drift) in square degrees.

Parameters

obsidint

The MWA observation ID.

common_metadatalist, optional

The list of common metadata generated from vcstools.metadb_utils.get_common_obs_metadata()

durint, optional

Duration of observation to calculate for in seconds. By default will use the entire observation duration.

Returns

areafloat

The field-of-view of the observation in square degrees.

vcstools.beam_calc.find_sources_in_obs(obsid_list, names_ra_dec, obs_for_source=False, dt_input=300, beam='analytic', min_z_power=0.3, cal_check=False, all_volt=False, degrees_check=False, metadata_list=None)

Either creates text files for each MWA obs ID of each source within it or a text file for each source with each MWA obs is that the source is in.

Parameters

obsid_listlist

List of MWA observation IDs.

names_ra_declist

An array in the format [[source_name, RAJ, DecJ]]

obs_for_sourceboolean, optional

If True creates a text file for each source with each MWA observation that the source is in. If False creates text files for each MWA obs ID of each source within it.
Default: False.

dt_inputint, optional

The time interval in seconds of how often powers are calculated.
Default: 300.

beamstr, optional

The primary beam model to use out of [analytic, advanced, full_EE].
Default: analytic.

min_z_powerfloat, optional

Zenith normalised power cut off.
Default: 0.3.

cal_checkboolean, optional

Checks the MWA pulsar database if there is a calibration suitable for the observation ID.

all_voltboolean, optional

Included observations with missing or incorrect voltage files.
Default: False.

degrees_checkboolean, optional

If true assumes RAJ and DecJ are in degrees.
Default: False.

metadata_listlist

List of the outputs of vcstools.metadb_utils.get_common_obs_metadata. If not provided, will make the metadata calls to find the data.
Default: None.

Returns

output_datadict

The format of output_data is dependant on obs_for_source.
If obs_for_source is True :
output_data = {jname:[[obsid, duration, enter, exit, max_power],
[obsid, duration, enter, exit, max_power]]}
If obs_for_source is False :
ouput_data = {obsid:[[jname, enter, exit, max_power],
[jname, enter, exit, max_power]]}

obsid_metalist

A list of the output of get_common_obs_metadata for each obsid

vcstools.beam_calc.from_power_to_gain(power, cfreq, n, coh=True)

Estimate the gain from the tile beam power.

Parameters

powerfloat

The tile beam power at the source position.

cfreqfloat

The centre frequency of the observation in Hz

nint

The number of non-flagged MWA tiles.

cohboolean, optional

True if the observation is coherent (tied-array beam) or False if it’s incoherent.
Default: True.

Returns

gainfloat

Gain in K/Jy.

vcstools.beam_calc.get_Trec(obsfreq, trcvr_file=None)

Get receiver temperature from the temperature receiver file.

Parameters

obsfreqfloat

The observing frequency in MHz.

trcvr_filestr, optional

The Trec file location to read in. If none is supplied, will use the Trec file in the data directory.

Returns

Trecfloat

The receiver temperature in K.

vcstools.beam_calc.get_beam_power_over_time(names_ra_dec, common_metadata=None, dt=296, centeronly=True, verbose=False, option='analytic', degrees=False, start_time=0)

Calculates the zenith normalised power for each source over time.

Parameters

names_ra_declist

An array in the format [[source_name, RAJ, DecJ]]

common_metadatalist, optional

The list of common metadata generated from vcstools.metadb_utils.get_common_obs_metadata()

dtint, optional

The time interval of how often powers are calculated.
Default: 296.

centeronlyboolean, optional

Only calculates for the centre frequency.
Default: True.

verboseboolean, optional

If True will not supress the output from mwa_pb.
Default: False.

optionstr, optional

The primary beam model to use out of [analytic, advanced, full_EE, hyperbeam].
Default: analytic.

degreesboolean, optional

If true assumes RAJ and DecJ are in degrees.
Default: False.

start_timeint, optional

The time in seconds from the begining of the observation to start calculating at.
Default: 0.

Returns

Powersnumpy.array, (len(names_ra_dec), ntimes, nfreqs)

The zenith normalised power for each source over time.

vcstools.beam_calc.pixel_area(ra_min, ra_max, dec_min, dec_max)

Calculate the area of a pixel on the sky from the pixel borders

Parameters

ra_minfloat

The Right Acension minimum in degrees.

ra_maxfloat

The Right Acension maximum in degrees.

dec_minfloat

The Declination minimum in degrees.

dec_maxfloat

The Declination maximum in degrees.

Returns

areafloat

Area of the pixel in square degrees.

vcstools.beam_calc.source_beam_coverage(obs_list, names_ra_dec, common_metadata_list=None, dt_input=300, beam='analytic', min_z_power=0.3)

For a list of MWA observations and sources will find if the sources are in the beam and when they enter and exit.

Parameters

obs_listlist

A list of MWA Observation IDs.

names_ra_declist

An array in the format [[source_name, RAJ, DecJ]]

common_metadata_listlist of list, optional

A list of lists where each list is the common metadata generated from vcstools.metadb_utils.get_common_obs_metadata() for each obs in the obs_list.

dt_inputint, optional

The time interval in seconds of how often powers are calculated.
Default: 300.

beamstr, optional

The primary beam model to use out of [analytic, advanced, full_EE, hyperbeam].
Default: analytic.

min_z_powerfloat, optional

Zenith normalised power cut off.
Default: 0.3.

Returns

beam_coveragedict

A dictionary where the first key is the observation ID and the second is the pulsar names like so:
beam_coverage[obsid][name] = [dect_beg_norm, dect_end_norm, np.amax(source_ob_power)] dect_beg_norm : float

Fraction of the observation when the source enters the beam.

dect_end_normfloat

Fraction of the observation when the source enters and exits the beam respectively.

vcstools.beam_calc.source_beam_coverage_and_times(obsid, pulsar, p_ra=None, p_dec=None, obs_beg=None, obs_end=None, files_beg=None, files_end=None, min_z_power=0.3, dt_input=100, common_metadata=None, query=None, beam='analytic')

Finds the normalised time that a pulsar is in the beam for a given obsid. If pulsar is not in beam, returns None, None

Parameters

obsidint

The observation ID

pulsarstr

The pulsar’s J name

p_ra, p_decstr, optional

The target’s right ascension and declination in sexidecimals. If not supplied will use the values from the ANTF.

obs_beg, obs_endint, optional

Beginning and end GPS time of the observation. If not supplied will use vcstools.metadb_utils.obs_max_min() to find it.

files_beg, files_endint, optional

Beginning and end GPS time of the (fits of VCS) files. If not supplied will assume the full observation is available.

min_z_powerfloat, optional

Zenith normalised power cut off.
Default: 0.3.

common_metadatalist, optional

The list of common metadata generated from vcstools.metadb_utils.get_common_obs_metadata()

querypsrqpy object, optional

A previous psrqpy query. Can be supplied to prevent performing a new query.

beamstr, optional

The primary beam model to use out of [analytic, advanced, full_EE].
Default: analytic.

Returns

enter_filesfloat

A float between 0 and 1 that describes the normalised time that the pulsar enters the beam

exit_filesfloat

A float between 0 and 1 that describes the normalised time that the pulsar exits the beam

beam_sim

The functions required to simulate the tied-array beam response of the MWA. All equations can be found in https://ui.adsabs.harvard.edu/abs/2018IAUS..337..378M/abstract

vcstools.beam_sim.cal_phase_ord(xpos, ypos, zpos, delays, gx, gy, gz, freq, coplanar=False, no_delays=False)

Equation 2 and 3 of Ord 2019.

Parameters

xpos[N]:

A list of tile positions East of the array centre

ypos[N]:

A list of tile positions North of the array centre

zpos[N]:

A list of tile heights about sea-level

gx[az/za]:

The x 3D wavenumbers for a given wavelength and az/za grid

gy[az/za]:

The y 3D wavenumbers for a given wavelength and az/za grid

gz[az/za]:

The z 3D wavenumbers for a given wavelength and az/za grid

Returns

ph_tile[N][az/za]:

A list of the phases for each tile

vcstools.beam_sim.calcArrayFactor(ph_tiles, ph_targets)

Calculates array factor pointed at some target zenith angle (za) and azimuth (az) (equation 11)

Parameters

ph_tiles[N][az/za]:

List of the sky phases for the tiles

ph_targets[N]:

List of the sky phases for the target

Returns

array_factor[az/za]:

The array factor for each za and az

array_factor_power[az/za]:

The array factor power for each za and az

vcstools.beam_sim.calcSkyPhase(xpos, ypos, zpos, kx, ky, kz, coplanar=False)

Completes the calculation of equation 7 to get the phase of the tiles for each position on the sky

Parameters

xposlist (N)

A list of tile positions East of the array centre

yposlist (N)

A list of tile positions North of the array centre

zposlist (N)

A list of tile heights about sea-level

kxlist (az/za)

The x 3D wavenumbers for a given wavelength and az/za grid

kylist (az/za)

The y 3D wavenumbers for a given wavelength and az/za grid

kzlist (az/za)

The z 3D wavenumbers for a given wavelength and az/za grid

Returns

ph_tilelist (N, az/za)

A list of the phases for each tile

vcstools.beam_sim.calcWaveNumbers(freq, p, t)

Function to calculate the 3D wavenumbers for a given wavelength and az/za grid. This is the part of equation 7 within the square brackets not includeing x_n, y_n and z_n

Parameters

freqfloat

Central frequency for the observation in Hz.

pfloat

azimuth/phi (either a scalar or an array) this is assuming that theta,phi are in the convention from Sutinjo et al. 2015.

tfloat

zenith angle/theta (either a scalar or an array).

Returns

kx, ky, kzlist

The 3D wavenumbers.

vcstools.beam_sim.calc_geometric_delay_distance(p, t)

Equation 1 of Ord 2019. Changed cos(el) to sin(za)

vcstools.beam_sim.calc_pixel_area(za, az_res, za_res)

Calculate the area of a pixel on the sky from their height, width and zenith angle

Parameters

zafloat

The zenith angle of the pixel in radians.

az_resfloat

The azuimuth resolution of the pixel in degrees.

za_resfloat

The zenith resolution of the pixel in degrees.

Returns

areafloat

Area of the pixel in square radians.

vcstools.beam_sim.getTileLocations(metafits, flags=None)

Function grab the MWA tile locations for a given observation from the metafits file.

Parameters

metafitsstr

The metafits file location.

flagslist, optional

RTS tile flags (i.e. the first entry in the metafits correspond to “tile 0”, irrespective of what the antenna name is).
Default: [].

Returns

listlist

A list of lists containing the following:

list[0] = a list of tile positions East of the array centre list[1] = a list of tile positions North of the array centre list[2] = a list of tile heights about sea-level list[3] = a list of tile cable delays

vcstools.beam_sim.get_obstime_duration(metafits)

Funciton to grab the recorded start-time and duration of the observation

Parameters

metafitsstr

The metafits file location.

Returns

obs_beginstr

Observation starting time in UTC.

obs_durationint

Observation duration in seconds.

vcstools.beam_sim.launch_pabeam_sim(obsid, pointing, begin, duration, source_name='noname', metafits_file=None, flagged_tiles=None, delays=None, efficiency=1, vcstools_version='master', args=None, common_metadata=None, output_dir=None)

Submit a job to run the pabeam code to estimate the system equivelent flux density and a dependent job to resume the submit_to_databse.py code if args is given.

Parameters

obsidint

The MWA observation ID.

pointingstr

The pointing of the simulation in the format HH:MM:SS.SS_DD:MM:SS.SS.

beginint

The begining of the simulation in GPS time.

durationint

The duration of the simulation in seconds (used to calculate the end of the simulation).

source_namestr, optional

The name of the source to be used to label output files.
Default: “noname”.

metafits_filestr, optional

The location of the metafits file. If none given will assume the default location.

flagged_tileslist, optional

A list of the flagged tiles. If none given will assume no tiles were flagged.

efficiencyfloat, optional

Frequency and pointing dependent array efficiency.
Default: 1.

vcstools_versionstr, optional

VCSTools version to load in the job.

argsdict, optional

The argument parse dictionary from submit_to_database.py. If supplied will launch a dependedn job with submit_to_databse.py to complete the script.

common_metadatalist, optional

The list of common metadata generated from vcstools.metadb_utils.get_common_obs_metadata().

output_dirstr

The output directory of the simulation results. By default will put it in the VCS directory under <obsid>/sefd_simulations.

Examples

A simple example:

>>>launch_pabeam_sim(1206977296, “12:49:12_+27:12:00”, 1206977300, 600, source_name=”SEFD_test”, output_dir=”.”)

vcstools.beam_sim.read_sefd_file(sefd_file, all_data=False)

Read in the output sefd file from a pabeam.py simulation.

Parameters

sefd_filestr

The location of the sefd file to be read in.

all_databoolean

If True will return the freq, sefd, t_sys, t_ant, gain, effective_area. If False will only return the sefd. Default False

catalogue_utils

vcstools.catalogue_utils.get_psrcat_dm_period(pulsar_list=None, query=None)

Uses PSRCAT to return a list of pulsar names, periods and dispersion measures.

Parameters

pulsar_listlist, optional

List of the pulsar Jnames to search the catalogue for. If no pulsar_list is given then returns all pulsar on the catalogue.
Default: None.

querypsrqpy object, optional

A previous psrqpy.QueryATNF query. Can be supplied to prevent performing a new query.

Returns

pulsar_dm_p: list

[[Jname, DM, period]]

Jnamestr

The Jname of the pulsar.

DMfloat

The Dispersion Measure of the pulsar.

periodfloat

The period of the puslsar in seconds.

vcstools.catalogue_utils.get_psrcat_ra_dec(pulsar_list=None, max_dm=5000.0, include_dm=False, query=None)

Uses PSRCAT to return a list of pulsar names, ras and decs. Not corrected for proper motion. Removes pulsars without any RA or DEC recorded

Parameters

pulsar_listlist, optional

List of the pulsar Jnames to search the catalogue for. If no pulsar_list is given then returns all pulsar on the catalogue.
Default: None.

max_dmfloat, optional

The maximum dispersion measure of pulsars to include in the output.
Default: 250.

include_dmboolean, optional

If True will also return the pulsars’ dispersion measure.
Default: False,

querypsrqpy object, optional

A previous psrqpy.QueryATNF query. Can be supplied to prevent performing a new query.

Returns

pulsar_ra_declist

[[Jname, RAJ, DecJ]]

Jnamestr

The Jname of the pulsar.

RAJstr

The Right Acension in the format “HH:MM:SS.SS”.

DecJstr

The Declination in the format “DD:MM:SS.SS”.

vcstools.catalogue_utils.get_rFRB_info(name=None)

Gets repeating FRB info from the csv file we maintain.

Parameters

namelist, optional

A list of repeating FRB names to get info for. The default is None which gets all rFRBs in the catalogue.

Returns

outputlist

[[name, RAJ, DecJ, dm, dm_error]]

namestr

The name of the source.

RAJstr

The Right Acension in the format “HH:MM:SS.SS”.

DecJstr

The Declination in the format “DD:MM:SS.SS”.

dmfloat

The Dispersion Measure of the pulsar.

dm_errorfloat

The uncertainty of the Dispersion Measure of the pulsar.

vcstools.catalogue_utils.grab_source_alog(source_type='Pulsar', pulsar_list=None, max_dm=5000.0, include_dm=False, query=None)

Will search different source catalogues and extract all the source names, RAs, Decs and, if requested, DMs

Parameters

source_typestr

The type of source you would like to get the catalogue for. Your choices are: [‘Pulsar’, ‘FRB’, ‘rFRB’, ‘POI’ ‘RRATs’, ‘Fermi’]
Default: ‘Pulsar’

pulsar_list: list

List of sources you would like to extract data for. If None is given then it will search for all available sources
Default: None

max_dmfloat

If the source_type is ‘Pulsar’ then you can set a maximum dm and the function will only return pulsars under that value.
Default: 1000.

include_dm: Bool

If True the function will also return the DM if it is available at the end of the list
Default: False

Returns

name_ra_declist

[[name, RAJ, DecJ, (DM)]]

namestr

The name of the source.

RAJstr

The Right Acension in the format “HH:MM:SS.SS”.

DecJstr

The Declination in the format “DD:MM:SS.SS”.

DMfloat

The Dispersion Measure of the pulsar. Only included if include_dm is True.

check_files

vcstools.check_files.check_download(obsID, directory=None, startsec=None, n_secs=None, data_type='raw')

Checks that the number of files in directory is the same as that found on the archive and also checks that all files have the same size (253440000 for raw, 7864340480 for recombined tarballs by default).

Parameters

obsIDint

The MWA Observation ID.

directorystr, optional

The directory in which to check the files.
Default: /astro/mwavcs/vcs/[obsID]/raw/.

startsecint

The gps time of first file to check.

n_secint

The number of seconds from the startsec to check.

data_typestr

The type of data from [‘raw’, ‘tar_ics’, ‘ics’].
Default: raw.

Returns

errorboolean

If True there are missing or incorrect files, False if all files are correct.

vcstools.check_files.check_recombine(obsID, directory=None, required_size=327680000, required_size_ics=30720000, startsec=None, n_secs=None)

Checks that the number of files in the directory as that found on the archive and also checks that all files have the same size (327680000 by default)

Parameters

obsIDint

The MWA Observation ID.

directorystr, optional

The directory in which to check the files.
Default: /astro/mwavcs/vcs/[obsID]/combined/.

required_sizeint

The required size of the recombined files in bytes.
Default: 327680000.

required_size_icsint

The required size of the ics files in bytes.
Default: 30720000.

startsecint

The gps time of first file to check.

n_secint

The number of seconds from the startsec to check.

Returns

errorboolean

If True there are missing or incorrect files, False if all files are correct.

vcstools.check_files.check_recombine_ics(obsID=None, directory=None, startsec=None, n_secs=None, required_size=None)

Checks that the number of recombined ics files in the directory as that found on the archive.

Parameters

obsIDint, optional

The MWA Observation ID.

directorystr, optional

The directory in which to check the files.
Default: /astro/mwavcs/vcs/[obsID]/combined/.

startsecint

The gps time of first file to check.

n_secint

The number of seconds from the startsec to check.

required_sizeint

The required size of the ics files in bytes.
Default: None.

Returns

errorboolean

If True there are missing or incorrect files, False if all files are correct.

vcstools.check_files.get_files_and_sizes(obsID, mode, mintime=0, maxtime=2000000000)

Get files and sizes from the MWA metadata server and check that they’re all the same size

Parameters

obsIDint

The MWA observation ID.

modestr

The typ of file from ‘raw’, ‘tar_ics’ and ‘ics’

mintimeint

The minimum GPS time of observations to check (inclusive, >=).
Default: 0

maxtimeint

The maximum GPS time of observations to check (exculsive, <).
Default: 2000000000

Returns

files_maskedlist

List of the files with the input mode/suffix.

suffixstr

The file suffix from [‘.dat’, ‘.tar’, ‘_ics.dat’] depnding on the input mode.

sizes[0]int

Size of files in bytes.

config

Functions to handle parsing the config file for multiple super computers

vcstools.config.load_config_file()

Work out which supercomputer you are using and load the appropriate config file

data_load

Loads all the data required by vcstools from the data directory.

general_utils

vcstools.general_utils.create_link(data_dir, target_dir, product_dir, link)

Creates a symbolic link product_dir/link that points to data_dir/target_dir.

Parameters

data_dirstr

The absolute path to the base directory of the true location of the files. For our uses this is often a scratch partition like /astro on Galaxy

target_dirstr

The folder you would like to be linked to

product_dirstr

The absolute path of the link you would like to create

linkstr

The name of the link you would like to create. Often the same as target_dir

vcstools.general_utils.is_number(s)

Simple is check to see if a string can be converted to an interger.

Parameters

sstr

String to check.

Returns

resultboolean

Boolean if it can be converted to an int.

vcstools.general_utils.mdir(path, description, gid=34858)

Simple function to create directories with the correct group permissions (771).

Parameters

pathstr

The path of the directory we want to create.

descriptionstr

The description of the directory to be printed to logger.

gidint, optional

The group ID to apply to the directory.
Default: 34858 which the mwavcs.

vcstools.general_utils.setup_logger(logger, log_level=20)

Setup the logger with the format we prefer and apply it to all import vcstools modules.

Parameters

loggerlogger object

The logger object to modify.

log_levellogging class

The logging level to apply.
Default: logging.INFO

Returns

loggerlogger object

The modified logger object.

vcstools.general_utils.sfreq(freqs)

Sore te coarse frequency channel IDs into the strange MWA format that reverses the order of channels above 128.

Parameters

freqslist

List of coarse frequency channel IDs.

Returns

freqslist

List of coarse frequency channel IDs in the new order.

vcstools.general_utils.split_remove_remainder(array, nchunks)

Split an array into nchunks and remove any remaining elements.

Parameters

arraynp.array, (N)

A single dimension array.

nchunksint

The number of sub arrays to split array into.

Returns

array_chunkslist

A list containing nhcunks arrays of equal size.

gfit

class vcstools.gfit.gfit(raw_profile, max_N=10, plot_name=None, component_plot_name=None, scattering_threshold=0.7, on_pulse_ranges=None)

This class is used to fit multiple Gaussians to a pulse profile.

Parameters

raw_profilelist

The pulse profile to be fit.

max_Nint

The maximum number of gaussians to attempt to fit.
Default: 10.

plot_namestr

The name of the output plot. Can be set with gfit.plot_name. If unsupplied, will use a generic name.
Default: None.

conponent_plot_namestr

The name of the plot for the component plot that illustrates how profile components are chosen. If unsupplied, will not plot. Only applicable if on_pulse_range unsupplied.
Default: None.

scattering_thresholdfloat

The threshold for which any tau (scattering pulse width) value greater will be deemed scattered (in phase).
Default: 0.7.

on_pulse_rangeslist

A list of two-lists/tuples that describes the on pulse region in phase. e.g. [[0.1, 0.2], [0.6, 0.7]] If not supplied, will attempt to find on-pulse region
Default: None.

Returns

fit_dictdict

contains the following keys:

W10float

The W10 width of the profile measured in phase.

W10_efloat

The uncertainty in the W10.

W50float

The W50 width of the profile measured in phase.

W50_efloat

The uncertainty in the W50.

Weqfloat

The equivalent width of the profile measured in phase.

Weq_efloat

The uncertainty in the equivalent width.

Wscatfloat

The scattering width of the profile measured in phaseprofile_region_from_pairs. The number of distinguishable profile components.

on_pulse_estimateslist

The output of prof_utils.estimate_components_onpulse(). Will be the rolled on_pulse_range input instead if supplied.

fit_paramslist

A list of length 5 + 3*(N-1) there N is num_gauss. Each set of 3 parameters corresponds to the amp, centre and width of a guassian component. The first five are baseline, tau, amp, centre and width.

cov_matnp.matrix

The covariance matrix output from curve_fit.

profilelist

The normalised and rolled input profile.

fitlist

The best fit made into a list form.

snfloat

The estimated signal to noise ratio, obtained from the profile.

sn_efloat

The uncertainty in sn.

scatteringfloat

The tau value of the profile fit.

scatteredboolean

Whether or not the final profile’s tau value is greater than the sattering threshold.

Attributes

component_plot_name

fit_dict

max_N

plot_name

Methods

auto_fit:	Runs a gaussian fit evaluation using up to max_N gaussians. The quality of the fit is decided via a chi-square evaluation. The best of these is saved and used to determine widths and maxima location. Part of the preprocessing of the profile will also determine the on-pulse regions and subsequently the noise level and signal to noise ratio.
plot_fit:	Plots the best chosen gaussian fit to a file whose name is gfit.plot_name. This can only be run after the fit_dict dictionary has been filled (presumably by auto_gfit).

auto_fit()

Fits multiple gaussian profiles and finds the best combination of N_Gaussians and alpha.

Uses

plot_fit()

Plots the best fit set of Gaussians.

Default output is “Gaussian_fit.png”.

job_submit

vcstools.job_submit.submit_slurm(name, commands, tmpl='{shebag}\n\n#SBATCH --export={export}\n#SBATCH --output={outfile}\n{account}\n#SBATCH --clusters={cluster}\n#SBATCH --partition={partition}\n#\n#SBATCH --cpus-per-task=1\n#SBATCH --ntasks-per-node={threads}\n#SBATCH --mem-per-cpu={mem}MB\n#SBATCH --nice={nice}\n{header}\n\nncpus={threads}\nexport OMP_NUM_THREADS={threads}\n\n{switches}\nmodule use {module_dir}\n{modules}\n\n{script}\n', slurm_kwargs=None, module_list=[], vcstools_version='master', batch_dir='batch/', depend=None, depend_type='afterok', submit=True, outfile=None, queue='cpuq', export='NONE', gpu_res=None, mem=1024, cpu_threads=1, temp_mem=None, nice=0, shebag='#!/bin/bash -l', module_dir=None, load_vcstools=True)

Making this function to cleanly submit SLURM jobs using a simple template.

Parameters

namestr

The base name that is used to create the “name.batch” and “name.out” files.

commandslist

Each item in the list is a line of the bash script commands you want to run.

tmplstr, optional

A template header string with format place holders: export, outfile, cluster, header and script. This is used to create the final string to be written to the job script. For this function, it is required to be SLURM compliant.
Default: SLURM_TMPL

slurm_kwargsdict, optional

A dictionary of SLURM keyword, value pairs to fill in whatever is not in the template supplied to tmpl.
Default: {} (empty dictionary, i.e. no additional header parameters).

module_listlist, optional

A list of module names (including versions if applicable) that will be included in the header for the batch scripts. e.g. [“vcstools/master”, “mwa-voltage/master”, “presto/master”] would append
module load vcstools/master
module load mwa-voltage/master
module load presto/master
to the header of the batch script. This can also invoke “module use …” commands. NOTE: /group/mwa/software/modulefiles is used and vcstools/master is loaded by default.

vcstools_versionstr, optional

The version of vcstools to load.
Default: master.

batch_dirstr, optional

The directory where you want to write the batch scripts
Default: “batch/”. (i.e. it will write to $PWD/batch_dir).

dependlist, optional

A list of the SLURM job IDs that your would like this job to depend on. If None then it is assumed there is no dependency on any other job.
Default: None.

depend_typestr, optional

The type of slurm dependancy required. For example if you wanted the job to run after the jobs have been terminated use ‘afterany’.
Default: “afterok”.

submitboolean, optional

Whether to write and submit the job scripts (True) or only write the scripts (False).
Default: True.

outfilestr, optional

The output file name if “name.out” is not desirable.
Default: None (i.e. “batch_dir/name.out”)

queuestr, optional

The type of queue you require (cpuq, gpuq or copyq) then the script will choose the correct partitions and clusters for the job to run on Default: “cpuq”

exportstr, optional

Switch that lets SLURM use your login environment on the compute nodes (“ALL”) or not (“NONE”).
Default: “None”.

gpu_resint, optional

Number of GPUs that the SLURM job will reserve.
Default: “None”.

memint, optional

The MB of ram required for your slurm job.
Default: 8192.

cpu_threadsint, optional

The number of CPU threads required for your slurm job.
Default: 1.

Returns

jobidint

The unique SLURM job ID associated with the submitted job.

metadb_utils

vcstools.metadb_utils.calc_ta_fwhm(freq, array_phase='P2C')

Calculates the approximate FWHM of the tied-array beam in degrees.

Parameters

freqfloat

Frequency in MHz.

array_phasestr, optional

The different array phase (from P1, P2C, P2E) to work out the maximum baseline length.
Default = ‘P2C’.

Returns

fwhmfloat

FWHM in degrees.

vcstools.metadb_utils.combined_deleted_check(obsid, begin=None, end=None)

Check if the combined files are deleted (or do not exist).

Parameters

obsidint

The MWA Observation ID.

begint, optional

Beginning of the observation GPS time to check.

endint, optional

End of the observation GPS time to check.

Returns

comb_del_checkbool

True if all combined files are deleted or if they do not exist.

vcstools.metadb_utils.ensure_metafits(data_dir, obsid, metafits_file)

Ensure that the metafits file is in the directory, if not download it.

Parameters

data_dirstr

The directory to check the metafits file is in.

obsidint

The MWA Observation ID.

metafits_filestr

The metafits file name.

vcstools.metadb_utils.files_available(obsid, files_meta_data=None)

Query the database and return a list of all files available (remote archived and not deleted) and a list of all files

Parameters

obsidint

The MWA Observation ID.

full_metadatadict, optional

The dictionary of metadata generated from vcstools.metadb_utils.getmeta()

Returns

available_fileslist

All files that have been archived and ready for download.

all_fileslist

All files that are expected, once all files have been transfered/archived.

vcstools.metadb_utils.find_obsids_meta_pages(params=None, pagesize=50)

Loops over pages for each page for MWA metadata calls

Parameters

paramsdict

The dictionary of constraints used to search for suitable observations as explained here: https://wiki.mwatelescope.org/display/MP/Web+Services#WebServices-Findobservations
Default: {‘mode’:’VOLTAGE_START’}

Returns

obsid_listlist

List of the MWA observation IDs.

vcstools.metadb_utils.get_ambient_temperature(obsid, full_metadata=None)

Queries the metadata to find the ambient temperature of the MWA tiles in K. If none recorded then assume 22.4 C.

Parameters

obsidint

The MWA observation ID.

full_metadatadict, optional

The dictionary of metadata generated from vcstools.metadb_utils.getmeta().

Returns

t_0float

The ambient temperature of the MWA tiles in K.

vcstools.metadb_utils.get_best_cal_obs(obsid)

For the input MWA observation ID find all calibration observations within 2 days that have the same observing channels and list them from closest in time to furthest.

Parameters

obsidint

The MWA Observation ID.

Returns

cal_idslist

[[obsid, mins_away, cal_target]]

obsidint

The MWA calibration observation ID.

mins_awayfloat

The mins away the calibration observation is away from the target observation.

cal_targetstr

The calibration target name.

vcstools.metadb_utils.get_channels(obsid, channels=None)

Gets the channels ids from the observation’s metadata. If channels is not None assumes the channels have already been aquired so it doesn’t do an unnecessary database call.

Parameters

obsidint

The MWA Observation ID.

channelslist, optional

The list of the coarse channel frequency IDs.

Returns

channelslist

The list of the coarse channel frequency IDs.

vcstools.metadb_utils.get_common_obs_metadata(obsid, return_all=False, full_metadata=None)

Gets needed common meta data from http://ws.mwatelescope.org/metadata/

Parameters

obsidint

The MWA observation ID.

return_allboolean, optional

If True will also return the full meta data dictionary.
Default: False.

full_metadatadict, optional

The dictionary of metadata generated from vcstools.metadb_utils.getmeta()

Returns

common_metadatalist

[obsid, ra, dec, dura, [xdelays, ydelays], centrefreq, channels]

obsidint

The MWA observation ID.

rastr

The Right Acension in the format “HH:MM:SS.SS”.

decstr

The Declination in the format “DD:MM:SS.SS”.

duraint

The duration of the observation in seconds.

xdelays, ydelayslist

The analogue delays for each antena od the tile for both the x and y polarisations.

centrefreqfloat

The centre observing frequency in MHz.

channelslist

The list of observing frequency coarse channel IDs.

vcstools.metadb_utils.get_files(obsid, files_meta_data=None)

Queries the metadata to find all the file names.

Parameters

obsidint

The MWA Observation ID.

full_metadatadict, optional

The dictionary of metadata generated from vcstools.metadb_utils.getmeta()

Returns

fileslist

A list of all the file names.

vcstools.metadb_utils.get_obs_array_phase(obsid)

For the input obsid will work out the observations array phase in the form of P1 for phase 1, P2C for phase 2 compact or P2E for phase to extended array and OTH for other.

Parameters

obsidint

The MWA Observation ID.

vcstools.metadb_utils.getmeta(servicetype='metadata', service='obs', params=None, retries=3, retry_http_error=False)

Function to call a JSON web service to perform an MWA metadata call. Taken verbatim from http://mwa-lfd.haystack.mit.edu/twiki/bin/view/Main/MetaDataWeb.

Parameters

servicetypestr

Either the ‘observation’ which makes human readable html pages or ‘metadata’ which returns data.
Default: metadata.

servicestr

The meta data service from (Defaul: obs):

obs:

Returns details about a single observation as explained in https://wiki.mwatelescope.org/display/MP/Web+Services#WebServices-Getobservation/scheduledataforanobservation

find:

Search the database for observations that satisfy given criteria as explained in https://wiki.mwatelescope.org/display/MP/Web+Services#WebServices-Findobservations

con:

Finds the configuration information for an observation as explained in https://wiki.mwatelescope.org/display/MP/Web+Services#WebServices-Gettelescopeconfigurationforagivenobservation

paramsdict

A dictionary of the options to use in the metadata call which is dependent on the service. Examples can be found https://wiki.mwatelescope.org/display/MP/Web+Services#WebServices-Usingthegetmeta()function

retriesint, optional

The number of times to retry timeout errors.

Returns

resultdict

The result for that service.

vcstools.metadb_utils.mwa_alt_az_za(obsid, ra=None, dec=None, degrees=False)

Calculate the altitude, azumith and zenith for an obsid

Parameters

obsidint

The MWA Observation ID.

rastr

The Right Acension in the format “HH:MM:SS.SS”.

decstr

The Declination in the format “DD:MM:SS.SS”.

degreesboolean

If True the ra and dec is given in degrees.
Default: False.

Returns

Altfloat

The altitude angle in degrees.

Azfloat

The azimuth angle in degrees.

Zafloat

The zenith angle in degrees.

vcstools.metadb_utils.obs_max_min(obsid, files_meta_data=None)

Small function to query the database and return the times of the first and last file.

Parameters

obsidint

The MWA Observation ID.

full_metadatadict, optional

The dictionary of metadata generated from vcstools.metadb_utils.getmeta()

Returns

obs_startint

Beginning of the observation GPS time.

obs_endint

End of the observation GPS time.

vcstools.metadb_utils.singles_source_search(ra, dec=None, box_size=50.0, params=None)

Used to find all obsids within a box around the source to make searching through obs_ids more efficient.

Parameters

rafloat

Right Acension of the source in degrees

decfloat

Declination of the source in degrees. By default will use the enitre declination range to account for grating lobes

box_sizefloat

Radius of the search box. Default: 45

paramsdict

The dictionary of constraints used to search for suitable observations as explained here: https://wiki.mwatelescope.org/display/MP/Web+Services#WebServices-Findobservations
Default: {‘mode’:’VOLTAGE_START’}

Returns

obsid_listlist

List of the MWA observation IDs.

pointing_utils

vcstools.pointing_utils.deg2sex(ra, dec)

Convert decimal coordingates into sexagesimal strings.

Parameters

rafloat

The Right Acension in degrees.

decfloat

The Declination in degrees.

Returns

rastr

The Right Acension in the format “HH:MM:SS.SS”.

decstr

The Declination in the format “DD:MM:SS.SS”.

vcstools.pointing_utils.format_ra_dec(ra_dec_list, ra_col=0, dec_col=1)

Will format a list of lists containing RAs and Decs to uniform strings. eg 00:00:00.00 -00:00:00.00. Will not work for numpy arrays so make sure they’re list of lists An example input: format_ra_dec([[name,ra,dec]], ra_col = 1, dec_col = 2)

Parameters

ra_dec_listlist of lists

A list of lists where each row is a different source with an RA and declination.

ra_colint, optional

The coloumn ID that contains the RA.
Default: 0.

dec_colint, optional

The coloumn ID that contains the Declination.
Default: 1.

Returns

ra_dec_listlist of lists

The formated ra_dec_list.

Examples

>>> format_ra_dec([["J2351+8533", "23:51:03", "+85:33:20.6"]], ra_col = 1, dec_col = 2)
[["J2351+8533", "23:51:03.00", "+85:33:20.60"]]

vcstools.pointing_utils.getTargetAZZA(ra, dec, time, lat=-26.7033, lon=116.671, height=377.827, units=(Unit('hourangle'), Unit('deg')))

Function to get the target position in alt/az at a given EarthLocation and Time. The default lat,lon,height is the centre of MWA.

Parameters

rastr

The target right ascension in astropy-readable format.

decstr

The target declination in astropy-readable format.

timestr

The time of observation in UTC (i.e. a string on form: yyyy-mm-dd hh:mm:ss.ssss)

latfloat, optional

The observatory latitude in degrees.
Default: -26.7033.

lonfloat, optional

The observatory longitude in degrees.
Default: 116.671.

heightfloat, optional

The observatory height from sea level in meters.
Default: 377.827.

unitstuple, optional

The astropy units of the ra and dec.
Default: (u.hourangle, u.deg)

Returns

azfloat

Target azimuth in radians.

zafloat

Target zenith angle in radians.

azdegfloat

Target azimuth in degrees.

zadegfloat

Target zenith angle in degrees.

vcstools.pointing_utils.getTargetRADec(az, za, time, lat=-26.7033, lon=116.671, height=377.827, units=(Unit('deg'), Unit('deg')))

Function to get the target position in ra dec at a given EarthLocation and Time. The default lat,lon,height is the centre of MWA.

Parameters

azstr

The target azimuth in astropy-readable format.

zastr

The target zenith angle in astropy-readable format.

timestr

The time of observation in UTC (i.e. a string on form: yyyy-mm-dd hh:mm:ss.ssss)

latfloat, optional

The observatory latitude in degrees.
Default: -26.7033.

lonfloat, optional

The observatory longitude in degrees.
Default: 116.671.

heightfloat, optional

The observatory height from sea level in meters.
Default: 377.827.

unitstuple, optional

The astropy units of the ra and dec.
Default: (u.deg, u.deg)

Returns

rafloat

Target right ascension in radians.

decfloat

Target declination in radians.

radegfloat

Target right ascension in degrees.

decdegfloat

Target declination in degrees.

vcstools.pointing_utils.sex2deg(ra, dec)

Convert sexagesimal coordinates to degrees.

Parameters

rastr

The Right Acension in the format “HH:MM:SS.SS”.

decstr

The Declination in the format “DD:MM:SS.SS”.

Returns

rafloat

The Right Acension in degrees.

decfloat

The Declination in degrees.

prof_utils

exception vcstools.prof_utils.BadFitError

Raise when a particular fit is not suitable

exception vcstools.prof_utils.LargeClipError

Raise when too much data is clipped

exception vcstools.prof_utils.LittleClipError

Raise when not enough data is clipped

exception vcstools.prof_utils.NoComponentsError

Raise when there are no feasible profile components

exception vcstools.prof_utils.NoFitError

Raise when no gaussian fits have been found

exception vcstools.prof_utils.ProfileLengthError

Raise when a profile’s legnth is too small to be fit

vcstools.prof_utils.analyse_pulse_prof(prof_data, period, alpha=3, scattering_threshold=0.7)

Estimates the signal to noise ratio and many other properties from a pulse profile.

Parameters

prof_datalist

A list of floats that contains the pulse profile.

periodfloat

The pulsar’s period in ms.

alphafloat, optional

The alpha value to use when clipping using vcstools.prof_utils.sigmaClip().
Default: 3.

Returns

prof_dictdict

The analysed profile’s dictionary which contains the following keys:

"sn"

The estimated signal to noise ratio (float).

"sn_e"

The estimated signal to noise ratio’s its uncertainty (float).

"profile"

A list containting the noramlised pulse profile (list).

"on_pulse_bool"

A list of booleans that are True if the profile is currently on the on pulse and False if the porifle is currently on the off pulse (list).

"Weq"

The equivalent width of the profile measured in phase (float).

"Weq_e"

The uncertaintiy in Weq_e (float).

"w_equiv_bins"

The equivalent width of the profile measured in bins (float).

"w_equiv_bins_e"

The uncertaintiy in w_equiv_bins (float).

"w_equiv_ms"

The equivalent width of the profile measured in ms (float).

"w_equiv_ms_e"

The uncertainty in w_equiv_ms (float).

"scattering"

The scattering width in phase (float).

"scattering_e"

The uncertainty in the scattering width in phase (float).

"scattered"

When true, the profile is highly scattered (boolean).

"flags"

A list of flagged data points (list).

vcstools.prof_utils.auto_analyse_pulse_prof(prof_data, period)

Automatically finds the best alpha value to use for analyse_pulse_prof() and returns the best resulting dictionary.

Parameters

prof_datalist

A list of floats that contains the pulse profile.

periodfloat

The period of the pulsar in ms.

Returns

fit_dictdict

The analysed profile’s dictionary which contains the following keys:

"sn"

The estimated signal to noise ratio (float).

"sn_e"

The estimated signal to noise ratio’s its uncertainty (float).

"flags"

A list of flagged data points (list).

"w_equiv_bins"

The equivalent width of the profile measured in bins (float).

"w_equiv_bins_e"

The uncertaintiy in w_equiv_bins (float).

"w_equiv_ms"

The equivalent width of the profile measured in ms (float).

"w_equiv_ms_e"

The uncertainty in w_equiv_ms (float).

"scattering"

The scattering width in ms (float).

"scattering_e"

The uncertainty in the scattering width in ms (float).

"scattered"

When true, the profile is highly scattered (boolean).

vcstools.prof_utils.check_clip(prof_to_check, toomuch=0.9, toolittle_frac=0.0, toolittle_absolute=4)

Determines whether a clipped profile from vcstools.prof_utils.sigmaClip() has been appropriately clipped by checking the number of nans. Raises a LittleClipError or a LargeClipError if too little or too much of the data has been clipped respectively.

Parameters

prof_to_checklist

The clipped profile from vcstools.prof_utils.sigmaClip()

toomuchfloat, optional

The fraction of the clipped profile beyond which is considered overclipped.
Default: 0.9

toolittlefloat, optional

The fraction of the clipped profile below which is considered underclipped.
Default: 0.

toolittle_absoluteint, optional

If a profile has this many or less on-pulse bins, it is deemed not sufficient.
Default: 4

vcstools.prof_utils.error_in_x_pos(x, y, sigma_y, x_pos)

Finds error in a position, x_pos, given an error in y derivatives are found numerically.

Parameters

xlist

A list that covers the x range (ie. np.linspace(0, 1, len(x))).

ylist

The y components of the x range.

sigma_yfloat

The absolute error in y.

x_posint

The location in x that we want to find the error for.

Returns

x_erfloat

The error in the x position.

vcstools.prof_utils.estimate_components_onpulse(profile, l=1e-05, plot_name=None)

Works by using stickel with lambda=1e-5 (default) to get a generic profile outline, then uses UnivariateSpline to find the min/maxima and estimate which of these are real depending on their relative amplitude. NOTE: The on-pulse estimates are read form LEFT to RIGHT. Meaning onpulse[0] can be a greater index than onpulse[1]. This indicates that the on-pulse is wrapped over the end of the phase.

Parameters

profilelist

The pulse profile.

lfloat, optional

The lambda value to use for Stickel regularisation. Don’t touch unless you know what you’re doing.
Default: 1e-5.

plot_namestr, optional

If supplied, will make a plot of the profile, its splined regularisation, maxima and best estimate of on-pulse.

Returns

comp_est_dictdict

Component estimation dictionary with keys:

"maxima"

The maximum points of the splined profile (list).

"underest_on_pulse"

List of tuples containing the understimated beginning and end of each profile component (list).

"overest_on_pulse"

List of tuples containing the overestimated beginning and end of each profile component (list).

"underest_off_pulse"

List of tuples containing the beginning and end of each off-pulse range from the underestimated on pulse (list).

"overest_off_pulse"

List of tuples containing the beginning and end of each off-pulse range from the overestimated on pulse (list).

"noise"

The standard deviation of the noise taken as the off-pulse region (float).

"alpha"

The alpha value def to vcstools.prof_utils.sigmaClip() to attain the initial on-pulse and noise estimate (float).

vcstools.prof_utils.filled_profile_region_between_pairs(profile, pairs, fill_value=0)

Fills the off-pair region of a profile with the fill value.

Parameters

profilelist

The profile we want to fill.

pairslist/tuple

A list of lists/tupls of length two that describes the region we DON’T want to fill.

fill_valuefloat, optional

The value to fill the region with.
Default=0.

vcstools.prof_utils.get_from_ascii(file_loc)

Retrieves the profile from an ascii file.

Parameters

file_locstr

The location of the ascii file.

Returns

profilelist

A list of floats containing the profile data.

len(profile)int

The number of bins in the profile.

vcstools.prof_utils.get_from_bestprof(file_loc, pointing_input=None)

Get info from a bestprof file

Parameters

file_locstr

The path to the bestprof file.

pointing_inputstr

If can not find a pointing in the bestprof file, will assume this is the pointing.
Default: None.

Returns

obsidint

The observation ID.

pulsarstr

The name of the pulsar.

dmfloat

The dispersion measure of the pulsar.

periodfloat

The period of the pulsar in ms.

period_uncerfloat

The uncertainty in the period measurement.

sigmafloat

The sigma (similar to signal to noise ratio).

obsstartint

The beginning time of the observation.

obslengthfloat

The length of the observation in seconds.

profile: list

A list of floats containing the profile data.

bin_numint

The number of bins in the profile.

vcstools.prof_utils.get_off_pulse(on_pulse_pairs)

Works out the off pulse ranges from the on-pulse.

Parameters

on_pulse_pairslist

List of sub-lists/tuples. Each sub-list is a pair of numbers that describes the upper and lower bounds of the ON PULSE region.

Returns

off_pulselist

A list of two-lists that decribes the off-pulse region.

vcstools.prof_utils.get_stokes_from_ascii(file_loc)

Retrieves the all stokes components from an ascii file.

Parameters

file_locstr

The location of the ascii file.

Returns

[I, Q, U, V, len(profile)]list

Ilist

Stokes I.

Qlist

Stokes Q.

Ulist

Stokes U.

Vlist

Stokes V.

len(profile)int

The number of bins in the profile.

vcstools.prof_utils.normamlise_prof(x)

Normalises a profile, x, to minimum 0 and maximum 1.

Parameters

xlist

A list of pulse profile values.

Returns

norm_xlist

A normalised profile.

vcstools.prof_utils.profile_region_from_pairs(profile, pairs)

Acquires a list of the region described by a pair of indexes accounting for wrap-around.

Parameters

profilelist

The profile that we want to get the region of.

pairslist/tuple

A list of lists/tuples of length two that describes the indexes of the region(s) to grab.

Returns

regionlist

The region of the profile described by the pairs.

vcstools.prof_utils.sigmaClip(data, alpha=3.0, tol=0.1, ntrials=10)

Sigma clipping operation: Compute the data’s median, m, and its standard deviation, sigma. Keep only the data that falls in the range (m-alpha*sigma,m+alpha*sigma) for some value of alpha, and discard everything else. This operation is repeated ntrials number of times or until the tolerance level is hit.

Parameters

datalist

A list of floats - the data to clip.

alphafloat, optional

Determines the number of sigmas to use to determine the upper and lower limits.
Default: 3.

tolfloat, optional

The fractional change in the standard deviation that determines when the tolerance is hit.
Default: 0.1.

ntrialsint, optional

The maximum number of times to apply the operation.
Default: 10.

Returns

oldstdfloat

The std of the clipped data.

xlist

The data list that contains only noise, with nans in place of ‘real’ data.

vcstools.prof_utils.sn_calc(profile, on_pulse_bool, noise_std, w_equiv_bins, sn_method='eqn_7.1')

Calculates the flux desnity using the siganl to noise (The maximum/peak signal divided by the STD of the noise) and the radiometer equation (see eqn A 1.21 of the pulsar handbook)

Parameters

profilelist

The normalised profile of the pulsar.

on_pulse_boollist

A list of booleans that are True if the profile is currently on the on pulse and False if the porifle is currently on the off pulse.

noise_stdfloat

The standard deviation of the noise in the normalised profile.

w_equiv_binsint

The equivalent width (in bins) of a top-hat pulse with the same are and peak height as the observed profile.

sn_methodstr, optional

The method of calculating the signal to noise ratio out of [“eqn_7.1”, “simple”]. Default “eqn_7.1”.

“Simple” uses 1/noise_std.

“eqn_7.1” uses equation 7.1 of the pulsar handbook.

Returns

snfloat

The signal to noise ratio of the profile.

u_snfloat

The uncertainty of the signal to noise ratio.

vcstools.prof_utils.subprocess_pdv(archive, outfile='archive.txt', pdvops='-FTt')

Runs the pdv commnand from PSRCHIVE as a python subprocess. NOTE: Requires singularity loaded in environment (module load singularity)

Parameters

archivestr

The name of the archive file to run the command on

outfilestr, optional

The name of the text file to write the output to.
Default: archive.txt.

pdvopsstr, optional

Additional options for the pdv.
Default: -FTt.

progress_bar

vcstools.progress_bar.progress_bar(it, prefix='', size=60, file=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>)

I stole this code from here: https://stackoverflow.com/questions/3160699/python-progress-bar

Parameters

itlist

The list to iterate over.

prefixstr

The prefix do display in the progress bar.
Default: “”.

sizeint

The length of the progress bar to display in characters.
Default: 60.

filestdout

The output of the progress bar.
Default: sys.stdout.

Examples

>>> for i in progressbar(range(15), "Computing: ", 40):

rm_synth

RM module that implements 1-D RM Synthesis/RMCLEAN.

MIT License

Copyright (c) 2017 George Heald

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

class vcstools.rm_synth.PolObservation(freq, IQU, IQUerr=None, verbose=True)

Class to describe an observation & perform polarimetry operations

Methods

get_fdf_peak([verbose])	Obtain the peak of the FDF and its Faraday depth This will fit the peak of the FSF and report results if requested.
plot_fdf([display, save, rescale, plot_rmsf])	Plot the FDF and RMSF
plot_stokesi([display, save])	Plot Stokes I
print_rmstats()	Print some stats about the results
rmclean([niter, gain, cutoff, mask, verbose])	Perform RMCLEAN
rmsynthesis(phi[, norm_mod, norm_vals, ...])	Perform RM Synthesis.

get_fdf_peak(verbose=True)

Obtain the peak of the FDF and its Faraday depth This will fit the peak of the FSF and report results if requested. Values that are calculated and reported are:
Absolute value at peak, PA at peak, peak RM value This is done for the dirty FDF. If RMCLEAN has been performed then this is also done for the deconvolved Faraday spectrum.

Parameters

verboseboolean, optional (default True)

Print some output?

plot_fdf(display=True, save=None, rescale=False, plot_rmsf=True)

Plot the FDF and RMSF

The plot will show the RMSF in black, and the FDF in red. If RMCLEAN has already been performed then the cleaned spectrum will be shown in blue, and the clean model in green.

Parameters

displayboolean, optional (default True)

Show plot on screen?

savestr, optional (default None)

Save figure to disk? Provide filename if desired.

rescaleboolean, optional (default False)

Rescale RMSF peak to match that of FDF?

plot_rmsfboolean, optional (default True)

Plot RMSF?

plot_stokesi(display=True, save=None)

Plot Stokes I

The plot will show the Stokes I values (and errors if available).

Parameters

displayboolean, optional (default True)

Show plot on screen?

savestr, optional (default None)

Save figure to disk? Provide filename if desired.

print_rmstats()

Print some stats about the results

Some basic statistics will be reported to the terminal:

mean of RM clean components, dispersion of RM clean components

rmclean(niter=1000, gain=0.1, cutoff=2.0, mask=False, verbose=True)

Perform RMCLEAN

The FDF will be deconvolved using the RMSF.

Parameters

niterint, optional (default 1000)

Maximum number of clean iterations

gainfloat, optional (default 0.1)

Clean gain

cutofffloat, optional (default 2)

Clean cutoff, in units of S/N The default stops at 2*sigma above the mean

maskboolean (default False)

If True, all clean components must be within an RMSF FWHM of the first peak

verboseboolean, optional (default True)

Print some output?

rmsynthesis(phi, norm_mod=False, norm_vals=False, double=True, clip=None, pclip=None, weightmode='none', verbose=True)

Perform RM Synthesis. This function performs RM Synthesis on the provided IQU data and computes the RMSF. It can adopt weights based on the IQU errors if they are provided.

Parameters

phiarray

RM values to use (should be contiguous and regularly spaced)

norm_modboolean, optional (default False)

Normalise QU values with the Stokes I fitted power-law model?

norm_valsboolean, optional (default False)

Normalise QU values with the Stokes I data points per channel?

doubleboolean, optional (default True)

Create the RMSF double the length along the RM axis? This should be kept as True unless you are not planning to RMCLEAN.

clipfloat, optional (default -inf)

Ignore channels with Stokes I S/N ratio < clip

pclipfloat, optional (default -inf)

Ignore channels with sqrt(Q**2+U**2) S/N ratio < pclip

weightmodestr, optional (default ‘none’)

How to do weighting in the Fourier transform. Options are:
‘none’ = no weighting (uniform weights)
‘varwt’ = inverse variance weights based on Stokes I noise Further options may be added later.

verboseboolean, optional (default True)

Print some output?

rm_synth_utils

vcstools.rm_synth_utils.IQU_rm_synth(freq_hz, I, Q, U, I_e, Q_e, U_e, phase_range=None, force_single=False, title=None, plotname=None, phi_range=(-300, 300), phi_steps=10000)

Performs RM synthesis on input data.

Parameters

freq_hzlist

Frequency values in Hz.

Ilist

Stokes I values.

I_elist

Stokes I uncertainties.

Qlist

Stokes Q values.

Q_elist

Stokes Q uncertainties.

Ulist

Stokes U values.

U_elist

Stokes U uncertainties.

phi_rangetuple, optional

The range of RMs to search over.
Default: (-300, 300).

phi_stepsint, optional

The bumber of RM steps to search over.
Default: 10000.

phase_rangetuple, optional

The phase range of the profile used in fitting. Will be displayed on plot.
Default: None.

plotnamestr, optional

The name of the plot. If None, will not plot:
Default: None.

Returns

rmfloat

The rotation measure.

rm_efloat

The uncertainty in the rotation measure.

vcstools.rm_synth_utils.find_best_range(I, Q, U, phase_ranges)

Finds the phase range with the largest ratio of lin pol to stokes I.

Parameters

Ilist

Stokes I.

Qlist

Stokes Q.

Ulist

Stokes U.

phase_rangeslist

A list where every 2 entries is a phase range from 0 to 1. ie. [0.1 0.3 0.4 0.8].

Returns

best_phase_rangelist

The most suitable phase range.

best_maxfloat

The maximum linear polarisation value.

vcstools.rm_synth_utils.find_on_pulse_ranges(I, clip_type='regular', plot_name=None)

Find ranges of pulse components from a pulse profile by fitting a gaussian distribution.

Parameters

Ilist

The pulse profile.

clip_typestr

The clipping verbosity for the Gaussian fitter. Choose between regular, noisy and verbose.

plot_namestr

The name of the ouput plot. If none, will not produce one.

Returns

phaseslist

A list of phases (from 0 to 1) corresponding to the on-pulse components.

vcstools.rm_synth_utils.read_rmfit_QUVflux(QUVflux)

Reads the freq, I, Q and U values with their errors from a QUVflux.out file generated from rmfit.

Parameters

QUVfluxstr

The QUVflux.out file

Returns

freq_hzlist

Frequency values in Hz.

Ilist

Stokes I values.

I_elist

Stokes I uncertainties.

Qlist

Stokes Q values.

Q_elist

Stokes Q uncertainties.

Ulist

Stokes U values.

U_elist

Stokes U uncertainties.

vcstools.rm_synth_utils.read_rmsynth_out(filename)

Reads the ouput file from vcstools.rm_synth_utils.rm_synth_pipe()

Parameters

filenamestr

The name of the file ouput from vcstools.rm_synth_utils.rm_synth_pipe().

Returns

rm_dictdict

Contains the following keys:

"i"

There are i entries where i is the number of different phase ranges (dict). Contains the following keys:

"rm"

The rotation measure of this run (float).

"rm_e"

The uncertainty in rm (float).

"phase_ranges"

The range of phases used for this run (tuple).

vcstools.rm_synth_utils.remove_allfiles(directory)

Delete all files in a directory then the directory itself.

Parameters

directorystr

The directory to delete.

vcstools.rm_synth_utils.rm_synth_pipe(kwargs)

Performs all the nexessary operations on an archive file to attain a rotation measure through the RM synthesis technique.

Parameters

archivestr

The name of the archive file to use.

work_dirstr, optional

The directory to work in.
Default: ‘./’.

pulsarstr, optional

The name of the puslar (used for naming purposes).
Default: None.

obsidint, optional

The observation ID (used for naming purposes).
Default: None.

plotboolean, optional

If True, will ouput a plot when finished.
Default: False.

writeboolean

If True, will write the result to a file.
Default: False.

labelstr, optional

A label used to identify the output files. If None, will generate a 64 bit string.

keep_QUVboolean, optional

If True, will keep the QUVflux.out file generated from rmfit.

force_singleboolean, optional

If True, will find the phase range with the greates lin_pol ratio and fit with only this (only if kwargs_rms[‘phase_range’] is None).
Default: False.

kwagrs_rmsdict

keyword arguments for vcstools.rm_synth_utils.IQU_rm_synth().

kwargs_gfitdict

keyword arguments for vcstools.prof_utils.auto_gfit().

Returns

rm_dict: dictionary

Contains the following keys:

"i"

There are i entries where i is the number of different phase ranges (dict). Contains the following keys:

"rm"

The rotation measure of this run (float).

"rm_e"

The uncertainty in rm (float).

"phase_ranges"

The range of phases used for this run (tuple).

"plotname"

The name of the output plot. None if no plot (str).

"label"

The label used (str).

filenamestr

The path of the file that was written to. None if not written.

vcstools.rm_synth_utils.rmfit_quad(archive, phase_min, phase_max)

Runs the PRSCHIVE rmfit command as a python subprocess using the -w option for a quadratic fit.

Parameters

archivestr

The name of the archive file to take as an input,

phase_minfloat

The minimum phase to begin the fit, should be a float between 0 and 1,

phase_maxfloat

The maximum phase to use to fit, should be a float between 0 and 1,

vcstools.rm_synth_utils.write_rm_to_yaml(filename, rm_dict)

Writes the rotation measure dictionary to a yaml file.

Parameters

filenamestr

The pathname of the file to write to.

rm_dictdict

A dictionary with the RM information formatted like the output from vcstools.rm_synth_utils.rm_synth_pipe().

sn_flux_utils

stickel