Wavelength Calibration API

class aspired.spectral_reduction.WavelengthCalibration(verbose=True, logger_name='WavelengthCalibration', log_level='INFO', log_file_folder='default', log_file_name=None)[source]

This is a wrapper for using RASCAL to perform wavelength calibration, which can handle arc lamps containing Xe, Cu, Ar, Hg, He, Th, Fe. This guarantees to provide something sensible or nothing at all. It will require some fine-tuning when using the first time. The more GOOD initial guesses provided, the faster the solution converges and with better fit. Knowing the dispersion, wavelength ranges and one or two known lines will significantly improve the fit. Conversely, wrong values supplied by the user will siginificantly distort the solution as any user supplied information will be treated as the ground truth.

Deatils of how RASCAL works should be referred to

Parameters:
  • verbose (bool (Default: True)) – Set to False to suppress all verbose warnings, except for critical failure.
  • logger_name (str (Default: OneDSpec)) – This will set the name of the logger, if the name is used already, it will reference to the existing logger. This will be the first part of the default log file name unless log_file_name is provided.
  • log_level (str (Default: 'INFO')) – Four levels of logging are available, in decreasing order of information and increasing order of severity: (1) DEBUG, (2) INFO, (3) WARNING, (4) ERROR and (5) CRITICAL. WARNING means that there is suboptimal operations in some parts of that step. ERROR means that the requested operation cannot be performed, but the software can handle it by either using the default setting or skipping the operation. CRITICAL means that the requested operation cannot be resolved without human interaction, this is most usually coming from missing data.
  • log_file_folder (None or str (Default: "default")) – Folder in which the file is save, set to default to save to the current path.
  • log_file_name (None or str (Default: None)) – File name of the log, set to None to self.logger.warning to screen only.
add_arc_lines(peaks)[source]

Provide the pixel locations of the arc lines.

Parameters:peaks (list) – The pixel locations of the arc lines. Multiple traces of the arc can be provided as list of list or list of arrays.
add_arc_spec(arc_spec)[source]

Provide the 1D spectrum of the arc image.

Parameters:arc_spec (list) – The photoelectron count of the 1D arc spectrum.
add_atlas(elements, min_atlas_wavelength=1000.0, max_atlas_wavelength=30000.0, min_intensity=10.0, min_distance=10.0, candidate_tolerance=10.0, constrain_poly=False, vacuum=False, pressure=101325.0, temperature=273.15, relative_humidity=0.0)[source]

Adds an atlas of arc lines to the calibrator, given an element.

Arc lines are taken from a general list of NIST lines and can be filtered using the minimum relative intensity (note this may not be accurate due to instrumental effects such as detector response, dichroics, etc) and minimum line separation.

Lines are filtered first by relative intensity, then by separation. This is to improve robustness in the case where there is a strong line very close to a weak line (which is within the separation limit).

The vacuum to air wavelength conversion is deafult to False because observatories usually provide the line lists in the respective air wavelength, as the corrections from temperature and humidity are small. See https://emtoolbox.nist.gov/Wavelength/Documentation.asp

Parameters:
  • elements (string or list of strings) – Chemical symbol, case insensitive
  • min_atlas_wavelength (float (Default: None)) – Minimum wavelength of the arc lines.
  • max_atlas_wavelength (float (Default: None)) – Maximum wavelength of the arc lines.
  • min_intensity (float (Default: None)) – Minimum intensity of the arc lines. Refer to NIST for the intensity.
  • min_distance (float (Default: None)) – Minimum separation between neighbouring arc lines.
  • candidate_tolerance (float (Default: 10)) – Tolerance (Angstroms) for considering a point to be an inlier during candidate peak/line selection. This should be reasonable small as we want to search for candidate points which are locally linear.
  • constrain_poly (bool (Default: Flase)) – Apply a polygonal constraint on possible peak/atlas pairs
  • vacuum (bool (Default: False)) – Set to True if the light path from the arc lamb to the detector plane is entirely in vacuum.
  • pressure (float (Default: 101325.)) – Pressure when the observation took place, in Pascal. If it is not known, assume 10% decrement per 1000 meter altitude
  • temperature (float (Default: 273.15)) – Temperature when the observation took place, in Kelvin.
  • relative_humidity (float (Default: 0.)) – In percentage.
add_fit_coeff(fit_coeff)[source]

Adding the polynomial coefficients.

Parameters:fit_coeff (list or list of list) – Polynomial fit coefficients.
add_fit_type(fit_type)[source]

Adding the polynomial type.

Parameters:fit_type (str or list of str) – Strings starting with ‘poly’, ‘leg’ or ‘cheb’ for polynomial, legendre and chebyshev fits. Case insensitive.
add_pix_wave_pair(pix, wave)[source]

Adding extra pixel-wavelength pair to the Calibrator for refitting. This DOES NOT work before the Calibrator having fit for a solution yet: use set_known_pairs() for that purpose.

Parameters:
  • pix (float) – pixel position
  • wave (float) – wavelength
add_user_atlas(elements, wavelengths, intensities=None, candidate_tolerance=10.0, constrain_poly=False, vacuum=False, pressure=101325.0, temperature=273.15, relative_humidity=0.0)[source]

Append the user supplied arc lines to the calibrator.

The vacuum to air wavelength conversion is deafult to False because observatories usually provide the line lists in the respective air wavelength, as the corrections from temperature and humidity are small. See https://emtoolbox.nist.gov/Wavelength/Documentation.asp

Parameters:
  • elements (list) – Element (required). Preferably a standard (i.e. periodic table) name for convenience with built-in atlases
  • wavelengths (list) – Wavelength to add (Angstrom)
  • intensities (list) – Relative line intensities
  • candidate_tolerance (float (Default: 10)) – Tolerance (Angstroms) for considering a point to be an inlier during candidate peak/line selection. This should be reasonable small as we want to search for candidate points which are locally linear.
  • constrain_poly (bool (Default: False)) – Apply a polygonal constraint on possible peak/atlas pairs
  • vacuum (bool (Default: False)) – Set to true to convert the input wavelength to air-wavelengths based on the given pressure, temperature and humidity.
  • pressure (float (Default: 101325.)) – Pressure when the observation took place, in Pascal. If it is not known, assume 10% decrement from 1 atm (the default) per 1000 meter altitude.
  • temperature (float (Default: 273.15)) – Temperature when the observation took place, in Kelvin.
  • relative_humidity (float (Default: 0.)) – In percentage.
clear_atlas()[source]

Remove all the lines loaded to the Calibrator.

do_hough_transform(brute_force=False)[source]

** brute_force is EXPERIMENTAL as of 1 Sept 2021 ** The brute force method is supposed to provide all the possible solution, hence given a sufficiently large max_tries, the solution should always be the best possible outcome. However, it does not seem to work in a small fraction of our tests. Use with caution, and it is not the recommended way for now.

Perform Hough transform on the pixel-wavelength pairs with the configuration set by the set_hough_properties().

Parameters:brute_force (bool (Default: False)) – Set to true to compute the gradient and intercept between every two data points
find_arc_lines(arc_spec=None, prominence=5.0, top_n_peaks=None, distance=5.0, refine=True, refine_window_width=5, display=False, renderer='default', width=1280, height=720, return_jsonstring=False, save_fig=False, fig_type='iframe+png', filename=None, open_iframe=False)[source]

This function identifies the arc lines (peaks) with scipy.signal.find_peaks(), where only the distance and the prominence keywords are used. Distance is the minimum separation between peaks, the default value is roughly twice the nyquist sampling rate (i.e. pixel size is 2.3 times smaller than the object that is being resolved, hence, the sepration between two clearly resolved peaks are ~5 pixels apart). A crude estimate of the background can exclude random noise which look like small peaks.

Parameters:
  • arc_spec (list, array or None (Default: None)) – If not provided, it will look for the arc_spec in the spectrum_oned. Otherwise, the input arc_spec will be used.
  • prominence (float (Default: 5.)) – The minimum prominence to be considered as a peak (% of max).
  • top_n_peaks (int (Default: None)) – The N most prominent peaks. None means keeping all peaks.
  • distance (float (Default: 5.)) – Minimum separation between peaks.
  • refine (bool (Default: True)) – Set to true to fit a gaussian to get the peak at sub-pixel precision.
  • refine_window_width (int (Default: 5)) – The number of pixels (on each side of the existing peaks) to be fitted with gaussian profiles over.
  • display (bool (Default: False)) – Set to True to display disgnostic plot.
  • renderer (string (Default: 'default')) – plotly renderer options.
  • width (int/float (Default: 1280)) – Number of pixels in the horizontal direction of the outputs.
  • height (int/float (Default: 720)) – Number of pixels in the vertical direction of the outputs.
  • return_jsonstring (bool (Default: False)) – set to True to return json string that can be rendered by Plotly in any support language.
  • save_fig (bool (default: False)) – Save an image if set to True. Plotly uses the pio.write_html() or pio.write_image(). The support format types should be provided in fig_type.
  • fig_type (string (default: 'iframe+png')) – Image type to be saved, choose from: jpg, png, svg, pdf and iframe. Delimiter is ‘+’.
  • filename (str (Default: None)) – Filename for the output, all of them will share the same name but will have different extension.
  • open_iframe (bool (Default: False)) – Open the iframe in the default browser if set to True. Only used if an iframe is saved.
Returns:

Return type:

JSON strings if return_jsonstring is set to True.

fit(max_tries=5000, fit_deg=4, fit_coeff=None, fit_tolerance=5.0, fit_type='poly', candidate_tolerance=2.0, brute_force=False, progress=True, return_jsonstring=False, display=False, renderer='default', save_fig=False, fig_type='iframe+png', filename=None, return_solution=True)[source]

A wrapper function to perform wavelength calibration with RASCAL. As of 14 January 2020, it supports He, Ne, Ar, Cu, Kr, Cd, Xe, Hg and Th from NIST.

Parameters:
  • max_tries (int) – Number of trials of polynomial fitting.
  • fit_deg (int (Default: 4)) – The degree of the polynomial to be fitted.
  • fit_coeff (list (Default: None)) – NOT CURRENTLY USED, as of 17 Jan 2021 Set the baseline of the least square fit. If no fits outform this set of polynomial coefficients, this will be used as the best fit.
  • fit_tolerance (float (Default: 5.0)) – Sets a tolerance on whether a fit found by RANSAC is considered acceptable.
  • fit_type (string (Default: 'poly')) – One of ‘poly’, ‘legendre’ or ‘chebyshev’.
  • candidate_tolerance (float (default: 2.0)) – toleranceold (Angstroms) for considering a point to be an inlier
  • brute_force (bool (Default: False)) – Set to True to try all possible combination in the given parameter space.
  • progress (bool (Default: True)) – Set to show the progress using tdqm (if imported).
  • return_jsonstring ((default: False)) – Set to True to save the plotly figure as json string.
  • display (bool (Default: False)) – Set to show diagnostic plot.
  • renderer (str (Default: 'default')) – plotly renderer options.
  • save_fig (string (Default: False)) – Set to save figure.
  • fig_type (string (default: 'iframe+png')) – Image type to be saved, choose from: jpg, png, svg, pdf and iframe. Delimiter is ‘+’.
  • filename (str) – Filename for the output, all of them will share the same name but will have different extension.
from_spectrum_oned(spectrum_oned, merge=False, overwrite=False)[source]

This function copies all the info from the spectrum_oned, because users may supply different level/combination of reduction, everything is copied from the spectrum_oned even though in most cases only a None will be passed.

By default, this is passing object by reference by default, so it directly modifies the spectrum_oned supplied. By setting merger to True, it copies the data into the SpectrumOneD in the FluxCalibration object.

Parameters:
  • spectrum_oned (SpectrumOneD object) – The SpectrumOneD to be referenced or copied.
  • merge (bool (Default: False)) – Set to True to copy everything over to the local SpectrumOneD, hence FluxCalibration will not be acting on the SpectrumOneD outside.
  • overwrite (bool (Default: False)) – Set to True to make a complete copy of the spectrum_oned to the target spectrum_oned, that includes all the Nones and other settings. Use with caution, as it removes the properties set before this function call.
get_calibrator()[source]

Get the calibrator object.

get_pix_wave_pairs()[source]

Return the list of matched_peaks and matched_atlas with their position in the array.

Returns:pw_pairs – List of tuples each containing the array position, peak (pixel) and atlas (wavelength).
Return type:list
get_spectrum_oned()[source]

Get the spectrum_oned object.

initialise_calibrator(peaks=None, arc_spec=None)[source]

Initialise a RASCAL calibrator.

Parameters:
  • peaks (list (Default: None)) – The pixel values of the peaks (start from zero).
  • arc_spec (list) – The spectral intensity as a function of pixel.
inspect_arc_lines(display=False, renderer='default', width=1280, height=720, return_jsonstring=False, save_fig=False, fig_type='iframe+png', filename=None, open_iframe=False)[source]
Parameters:
  • display (bool (Default: False)) – Set to True to display disgnostic plot.
  • renderer (string (Default: 'default')) – plotly renderer options.
  • width (int/float (Default: 1280)) – Number of pixels in the horizontal direction of the outputs.
  • height (int/float (Default: 720)) – Number of pixels in the vertical direction of the outputs.
  • return_jsonstring (bool (Default: False)) – set to True to return json string that can be rendered by Plotly in any support language.
  • save_fig (bool (default: False)) – Save an image if set to True. Plotly uses the pio.write_html() or pio.write_image(). The support format types should be provided in fig_type.
  • fig_type (string (default: 'iframe+png')) – Image type to be saved, choose from: jpg, png, svg, pdf and iframe. Delimiter is ‘+’.
  • filename (str (Default: None)) – Filename for the output, all of them will share the same name but will have different extension.
  • open_iframe (bool (Default: False)) – Open the iframe in the default browser if set to True. Only used if an iframe is saved.
Returns:

Return type:

JSON strings if return_jsonstring is set to True.

list_atlas()[source]

List all the lines loaded to the Calibrator.

manual_refit(matched_peaks=None, matched_atlas=None, degree=None, x0=None, return_solution=True)[source]

Perform a refinement of the matched peaks and atlas lines.

This function takes lists of matched peaks and atlases, along with user-specified lists of lines to add/remove from the lists.

Any given peaks or atlas lines to remove are selected within a user-specified tolerance, by default 1 pixel and 5 atlas Angstrom.

The final set of matching peaks/lines is then matched using a robust polyfit of the desired degree. Optionally, an initial fit x0 can be provided to condition the optimiser.

The parameters are identical in the format in the fit() and match_peaks() functions, however, with manual changes to the lists of peaks and atlas, peak_utilisation and atlas_utilisation are meaningless so this function does not return in the same format.

Parameters:
  • matched_peaks (list (Default: None)) – List of matched peaks
  • matched_atlas (list (Default: None)) – List of matched atlas lines
  • degree (int (Default: None)) – Polynomial fit degree (Only used if x0 is None)
  • x0 (list (Default: None)) – Initial fit coefficients
  • return_solution (bool (Default: True)) – Set to True to return the best fit polynomial coefficients.
plot_search_space(fit_coeff=None, top_n_candidate=3, weighted=True, save_fig=False, fig_type='iframe+png', filename=None, return_jsonstring=False, renderer='default', display=False)[source]

A wrapper function to plot the search space in the Hough space.

If fit fit_coefficients are provided, the model solution will be overplotted.

Parameters:
  • fit_coeff (list (default: None)) – List of best polynomial fit_coefficients
  • top_n_candidate (int (default: 3)) – Top ranked lines to be fitted.
  • weighted ((default: True)) – Draw sample based on the distance from the matched known wavelength of the atlas.
  • save_fig (bool (default: False)) – Save an image if set to True. matplotlib uses the pyplot.save_fig() while the plotly uses the pio.write_html() or pio.write_image(). The support format types should be provided in fig_type.
  • fig_type (string (default: 'iframe+png')) – Image type to be saved, choose from: jpg, png, svg, pdf and iframe. Delimiter is ‘+’.
  • filename ((default: None)) – The destination to save the image.
  • return_jsonstring ((default: False)) – Set to True to save the plotly figure as json string.
  • renderer ((default: 'default')) – Set the rendered for the plotly display. Ignored if matplotlib is used.
  • display (bool (Default: False)) – Set to True to display disgnostic plot.
Returns:

Return type:

json object if json is True.

remove_arc_lines()[source]

Remove all the refined arc lines.

remove_arc_spec()[source]

Remove the aspectrm of the arc

remove_atlas_lines_range(wavelength, tolerance=10.0)[source]

Remove arc lines within the given wavelength range (tolerance).

Parameters:
  • wavelength (float) – Wavelength to remove (Angstrom)
  • tolerance (float) – Tolerance around this wavelength where atlas lines will be removed
remove_fit_coeff()[source]

To remove the polynomial fit coefficients.

remove_fit_type()[source]

To remove the polynomial fit type.

remove_pix_wave_pair(arg)[source]

Remove fitted pixel-wavelength pair from the Calibrator for refitting. The positions can be found from get_pix_wave_pairs(). One at a time.

Parameters:arg (int) – The position of the pairs in the arrays.
robust_refit(fit_coeff, n_delta=None, refine=False, tolerance=10.0, method='Nelder-Mead', convergence=1e-06, robust_refit=True, fit_deg=None, display=False, renderer='default', filename=None, return_jsonstring=False, save_fig=False, fig_type='iframe+png', return_solution=True)[source]

** refine option is EXPERIMENTAL, as of 17 Jan 2021 ** A wrapper function to robustly refit the wavelength solution with RASCAL when there is already a set of good coefficienes.

Refine the polynomial fit coefficients. Recommended to use in it multiple calls to first refine the lowest order and gradually increase the order of coefficients to be included for refinement. This is be achieved by providing delta in the length matching the number of the lowest degrees to be refined.

Set refine to True to improve on the polynomial solution.

Set robust_refit to True to fit all the detected peaks with the given polynomial solution for a fit using maximal information, with the degree of polynomial = fit_deg.

Set both refine and robust_refit to False will return the list of arc lines are well fitted by the current solution within the tolerance limit provided.

Parameters:
  • fit_coeff (list) – List of polynomial fit coefficients.
  • n_delta (int (Default: None)) – The number of the highest polynomial order to be adjusted
  • refine (bool (Default: False)) – Set to True to refine solution.
  • tolerance (float (Default: 10.)) – Absolute difference between fit and model in the unit of nm.
  • method (string (Default: 'Nelder-Mead')) – scipy.optimize.minimize method.
  • convergence (float (Default: 1e-6)) – scipy.optimize.minimize tol.
  • robust_refit (bool (Default: True)) – Set to True to fit all the detected peaks with the given polynomial solution.
  • fit_deg (int (Default: length of the input coefficients)) – Order of polynomial fit with all the detected peaks.
  • display (bool (Default: False)) – Set to show diagnostic plot.
  • renderer (str (Default: 'default')) – plotly renderer options.
  • save_fig (string (Default: False)) – Set to save figure.
  • fig_type (string (default: 'iframe+png')) – Image type to be saved, choose from: jpg, png, svg, pdf and iframe. Delimiter is ‘+’.
  • filename (str (Default: None)) – Filename for the output, all of them will share the same name but will have different extension.
  • return_solution (bool (Default: True)) – Set to True to return the best fit polynomial coefficients.
save_csv(output='wavecal', filename='wavecal', overwrite=False, recreate=False)[source]

Save the reduced data to disk, with a choice of any combination of the data that are already present in the SpectrumOneD. Because a WavelengthCalibration only requires a subset of all the data, only ‘wavecal’ is guaranteed to exist.

Parameters:
  • output (String) –

    Type of data to be saved, the order is fixed (in the order of the following description), but the options are flexible. The input strings are delimited by “+”,

    wavecal: 1 HDU
    Polynomial coefficients for wavelength calibration
  • filename (String) – Disk location to be written to. Default is at where the process/subprocess is execuated.
  • overwrite (bool) – Default is False.
  • recreate (bool (Default: False)) – Set to True to overwrite the FITS data and header.
save_fits(output='wavecal', filename='wavecal', overwrite=False, recreate=False, empty_primary_hdu=True)[source]

Save the reduced data to disk, with a choice of any combination of the data that are already present in the SpectrumOneD. Because a WavelengthCalibration only requires a subset of all the data, only ‘wavecal’ is guaranteed to exist.

Parameters:
  • output (String) –

    Type of data to be saved, the order is fixed (in the order of the following description), but the options are flexible. The input strings are delimited by “+”,

    wavecal: 1 HDU
    Polynomial coefficients for wavelength calibration
  • filename (String) – Disk location to be written to. Default is at where the process/subprocess is execuated.
  • overwrite (bool) – Default is False.
  • recreate (bool (Default: False)) – Set to True to overwrite the FITS data and header.
  • empty_primary_hdu (bool (Default: True)) – Set to True to leave the Primary HDU blank
set_calibrator_properties(num_pix=None, pixel_list=None, plotting_library='plotly', logger_name='Calibrator', log_level='info')[source]

Set the properties of the calibrator.

Parameters:
  • num_pix (int (Default: None)) – The number of pixels in the dispersion direction
  • pixel_list (list or numpy array (Default: None)) – The pixel position of the trace in the dispersion direction. This should be provided if you wish to override the default range(num_pix), for example, in the case of accounting for chip gaps (10 pixels) in a 3-CCD setting, you should provide [0,1,2,…90, 100,101,…190, 200,201,…290]
  • plotting_library (string (Default: 'plotly')) – Choose between matplotlib and plotly.
  • log_level (string (Default: 'info')) – Choose from {CRITICAL, ERROR, WARNING, INFO, DEBUG, NOTSET}.
set_hough_properties(num_slopes=5000, xbins=500, ybins=500, min_wavelength=3000, max_wavelength=9000, range_tolerance=500, linearity_tolerance=100)[source]

Set the properties of the hough transform.

Parameters:
  • num_slopes (int (Default: 1000)) – Number of slopes to consider during Hough transform
  • xbins (int (Default: 50)) – Number of bins for Hough accumulation
  • ybins (int (Default: 50)) – Number of bins for Hough accumulation
  • min_wavelength (float (Default: 3000)) – Minimum wavelength of the spectrum.
  • max_wavelength (float (Default: 9000)) – Maximum wavelength of the spectrum.
  • range_tolerance (float (Default: 500)) – Estimation of the error on the provided spectral range e.g. 3000-5000 with tolerance 500 will search for solutions that may satisfy 2500-5500
  • linearity_tolerance (float (Default: 100)) – A toleranceold (Ansgtroms) which defines some padding around the range tolerance to allow for non-linearity. This should be the maximum expected excursion from linearity.
set_known_pairs(pix=None, wave=None)[source]

Provide manual pixel-wavelength pair(s), they will be appended to the list of pixel-wavelength pairs after the random sample being drawn from the RANSAC step, i.e. they are ALWAYS PRESENT in the fitting step. Use with caution because it can skew or bias the fit significantly, make sure the pixel value is accurate to at least 1/10 of a pixel.

This can be used, for example, for low intensity lines at the edge of the spectrum.

Parameters:
  • pix (numeric value, list or numpy 1D array (N) (Default: None)) – Any pixel value, can be outside the detector chip and serve purely as anchor points.
  • wave (numeric value, list or numpy 1D array (N) (Default: None)) – The matching wavelength for each of the pix.
set_ransac_properties(sample_size=5, top_n_candidate=5, linear=True, filter_close=False, ransac_tolerance=5, candidate_weighted=True, hough_weight=1.0, minimum_matches=3, minimum_peak_utilisation=0.0, minimum_fit_error=0.0001)[source]

Set the properties of the RANSAC process.

Parameters:
  • sample_size (int (Default: 5)) – Number of pixel-wavelength hough pairs to be used for each arc line being picked.
  • top_n_candidate (int (Default: 5)) – Top ranked lines to be fitted.
  • linear (bool (Default: True)) – True to use the hough transformed gradient, otherwise, use the known polynomial.
  • filter_close (bool (Default: False)) – Remove the pairs that are out of bounds in the hough space.
  • ransac_tolerance (float (Default: 1)) – The distance criteria (Angstroms) to be considered an inlier to a fit. This should be close to the size of the expected residuals on the final fit (e.g. 1A is typical)
  • candidate_weighted (bool (Default: True)) – Set to True to down-weight pairs that are far from the fit.
  • hough_weight (float or None (Default: 1.0)) – Set to use the hough space to weigh the fit. The theoretical optimal weighting is unclear. The larger the value, the heavily it relies on the overdensity in the hough space for a good fit.
  • minimum_matches (int (Default: 3)) – Minimum number of fitted peaks to accept as a solution. This has to be smaller than or equal to the sample size.
  • minimum_peak_utilisation (float (Default: 0.)) – The minimum percentage of peaks used in order to accept as a valid solution.
  • minimum_fit_error (float (Default 1e-4)) – Set to remove overfitted/unrealistic fits.