INS Class

This is the INS (incoherent_noise_spectrum) class. This is the main product that RFI analysis is performed on.

class SSINS.INS(indata=None, history='', label='', run_check=True, check_extra=True, run_check_acceptability=True, order=0, mask_file=None, match_events_file=None, spectrum_type='cross', use_integration_weights=False, nsample_default=1, **kwargs)

Defines the incoherent noise spectrum (INS) class, which is a subclass of the UVFlag class, a member of the pyuvdata software package. Below is a list of attributes that are either specific to the INS object, or attributes that also exist on UVFlag objects but have special meaning for this subclass. See the pyuvdata documentation for other UVFlag attributes that may be useful for data analysis.

metric_array

Array that contains the raw incoherently averaged noise spectrum data. Shape (Ntimes, Nfreqs, Npols).

Type:

masked_array

metric_ms

An array containing the z-scores of the data in the metric_array. Shape (Ntimes, Nfreqs, Npols).

Type:

masked_array

match_events

A list of tuples that contain information about events caught during match filtering.

Type:

list

weights_array

Array containing the sum of the weights down the baseline axis of the original SS object. Shape (Ntimes, Nfreqs, Npols).

Type:

array

weights_square_array

Array containing the sum of the squares of the weights down the baseline axis of the original SS object (for variance normalization). Shape (Ntimes, Nfreqs, Npols).

Type:

array

history

String containing the history of the data.

Type:

str

sig_array

An array that is initially equal to the z-score of each data point. During flagging, the entries are assigned according to their z-score at the time of their flagging.

Type:

array

spectrum_type

‘cross’ or ‘auto’, indicating the type of visibilities the spectrum was made from.

Type:

str

order

The order of polynomial fit for each frequency channel during mean-subtraction. Default is 0, which just calculates the mean.

Type:

int

use_integration_weights

Whether the product of the integration_array and nsample_array of the SS object were used as weights. If False (default), just use uniform weights based on the mask of the SS object’s data array.

Type:

bool

nsample_default

The default nsample value to fill zeros in the nsample_array with when there are some nsample=0. Important when working with data from uvfits files, which combine information from the flag_array and nsample_array in the weights field of the uvfits file.

Type:

float

mask_file

Path to a mask_file for filling out the initial mask of the metric_array.

Type:

str

match_events_file

Path to yaml file to fill out match_events attribute with previously found RFI events.

Type:

str

flag_uvf(uvf, inplace=False)

Applies flags calculated from mask_to_flags method onto a given UVFlag object. Option to edit an existing uvf object inplace. Works by propagating the mask on sky-subtracted data to flags that can be applied to the original data, pre-subtraction. ORs the flags from the INS object and the input uvf object.

Parameters:

• uvf – A waterfall UVFlag object in flag mode to apply flags to. Must be constructed from the original data. Errors if not waterfall, in flag mode, or time ordering does not match INS object.

• inplace – Whether to edit the uvf input inplace or not. Default False.

Returns:

The UVFlag object in flag mode with the time-propagated flags.

Return type:

uvf

from_uvdata(indata, mode='metric', copy_flags=False, waterfall=False, history='', label='', run_check=True, check_extra=True, run_check_acceptability=True, **kwargs)

Construct an INS object from a UVData (SS) object. This is called during instantiation, but due to inheritance

issues, is not implemented in such a way as to allow overwriting an already instantiated object (as is possible with a UVFlag object, the parent class for the INS object).

Parameters:

• indata (SS) – An SS object from which to construct the INS object.

• mode (str) – Does nothing – for compatibility with base class.

• copy_flags (bool) – Does nothing – for compatibility with base class.

• waterfall (bool) – Does nothing – for compatibility with base class.

• history (str) – History to be appended to history string of object.

• run_check (bool) – Whether to check that the object’s parameters have the right shape (default True).

• check_extra (bool) – Whether to also check optional parameters (default True)

• run_check_acceptability (bool) – Whether to check that the object’s parameters take appropriate values (default True).

• **kwargs – keywords to pass to UVFlag.from_uvdata – used for more reliable future compatibility with pyuvdata

mask_to_flags()

Propagates the mask to construct flags for the original (non time-differenced) data. If a time is flagged in the INS, then both times that could have contributed to that time in the sky-subtraction step are flagged in the new array.

Returns:

The time-propagated flags

Return type:

tp_flags (array)

match_events_read(filename)

Reads match events from file specified by filename argument

Parameters:

filename – The yml file with the stored match_events

Returns:

The match_events in the yml file

Return type:

match_events

mean_subtract(freq_slice=slice(None, None, None), return_coeffs=False)

A function which calculated the mean-subtracted spectrum from the regular spectrum. A spectrum made from a perfectly clean observation will be written as a z-score by this operation.

Parameters:

• freq_slice – The frequency slice over which to do the calculation. Usually not set by the user.

• return_coeffs – Whether or not to return the mean/polynomial coefficients

Returns:

The mean-subtracted data array.

Return type:

MS (masked array)

read(filename, history='', run_check=True, check_extra=True, run_check_acceptability=True, **kwargs)

Populate the object by reading a file. This is called during instantiation, but due to inheritance issues, is not

implemented in such a way as to allow overwriting a previously instantiated object (as is possible with a UVFlag object, the parent class for the INS object).

Parameters:

• filename (str) – Path to the file to be read.

• history (str) – History to be appended to the object’s history string.

• run_check (bool) – Whether to check that the object’s parameters have the right shape (default True).

• check_extra (bool) – Whether to also check optional parameters (default True)

• run_check_acceptability (bool) – Whether to check that the object’s parameters take appropriate values (default True).

• **kwargs – keywords to pass to UVFlag.read – used for more reliable future compatibility with pyuvdata

Returns:

select(inplace=True, **kwargs)

Thin wrapper around UVFlag.select that also recalculates the ms array immediately afterwards.

Parameters:

inplace – Whether to do the operation inplace or return a copy without touching the original.

Returns:

INS object. Only returned if inplace is False.

Return type:

ins

set_extra_params(order=0, spectrum_type='cross', use_integration_weights=False, nsample_default=1, mask_file=None, match_events_file=None)

Set non-datalike required parameters that are not inherited from UVFlag.

Parameters:

order – Sets the order or polynomial used when doing mean subtraction. Setting to 0 (default) just does a mean subtraction

set_ins_data_params()

Set special parameters specific to INS object that are not included in parent UVFlag object.

write(prefix, clobber=False, data_compression='lzf', output_type='data', mwaf_files=None, mwaf_method='add', metafits_file=None, Ncoarse=24, sep='_', uvf=None)

Writes attributes specified by output_type argument to appropriate files with a prefix given by prefix argument. Can write mwaf files if required mwaf keywords arguments are provided. Required mwaf keywords are not required for any other purpose.

Parameters:

• prefix – The filepath prefix for the output file e.g. /analysis/SSINS_outdir/obsid

• clobber – See UVFlag documentation

• data_compression – See UVFlag documentation

• output_type ('data', 'z_score', 'mask', 'flags', 'match_events') –

  data - outputs the metric_array attribute into an h5 file

  z_score - outputs the the metric_ms attribute into an h5 file

  mask - outputs the mask for the metric_array attribute into an h5 file

  flags - converts mask to flag using mask_to_flag() method and writes to an h5 file readable by UVFlag

  match_events - Writes the match_events attribute out to a human-readable yml file

  mwaf - Writes an mwaf file by converting mask to flags.

• mwaf_files (seq) – A list of paths to mwaf files to use as input for each coarse channel

• mwaf_method ('add' or 'replace') – Choose whether to add SSINS flags to current flags in input file or replace them entirely

• metafits_file (str) – A path to the metafits file if writing mwaf outputs. Required only if writing mwaf files.

• sep (str) – Determines the separator in the filename of the output file.