sherlock.transient_classifier module

classify a set of transients defined by a database query in the sherlock settings file

Author

David Young

class transient_classifier(log, settings=False, update=False, ra=False, dec=False, name=False, verbose=0, updateNed=True, daemonMode=False, updatePeakMags=True, oneRun=False, lite=False)[source]

Bases: object

The Sherlock Transient Classifier

Key Arguments

  • log – logger

  • settings – the settings dictionary

  • update – update the transient database with crossmatch results (boolean)

  • ra – right ascension of a single transient source. Default False

  • dec – declination of a single transient source. Default False

  • name – the ID of a single transient source. Default False

  • verbose – amount of details to print about crossmatches to stdout. 0|1|2 Default 0

  • updateNed – update the local NED database before running the classifier. Classification will not be as accuracte the NED database is not up-to-date. Default True.

  • daemonMode – run sherlock in daemon mode. In daemon mode sherlock remains live and classifies sources as they come into the database. Default True

  • updatePeakMags – update peak magnitudes in human-readable annotation of objects (can take some time - best to run occationally)

  • lite – return only a lite version of the results with the topped ranked matches only. Default False

  • oneRun – only process one batch of transients, usful for unit testing. Default False

Usage

To setup your logger, settings and database connections, please use the fundamentals package (see tutorial here).

To initiate a transient_classifier object, use the following:

Todo

  • update the package tutorial if needed

The sherlock classifier can be run in one of two ways. The first is to pass into the coordinates of an object you wish to classify:

from sherlock import transient_classifier
classifier = transient_classifier(
    log=log,
    settings=settings,
    ra="08:57:57.19",
    dec="+43:25:44.1",
    name="PS17gx",
    verbose=0
)
classifications, crossmatches = classifier.classify()

The crossmatches returned are a list of dictionaries giving details of the crossmatched sources. The classifications returned are a list of classifications resulting from these crossmatches. The lists are ordered from most to least likely classification and the indicies for the crossmatch and the classification lists are synced.

The second way to run the classifier is to not pass in a coordinate set and therefore cause sherlock to run the classifier on the transient database referenced in the sherlock settings file:

from sherlock import transient_classifier
classifier = transient_classifier(
    log=log,
    settings=settings,
    update=True
)
classifier.classify()

Here the transient list is selected out of the database using the transient query value in the settings file:

database settings:
    transients:
        user: myusername
        password: mypassword
        db: nice_transients
        host: 127.0.0.1
        transient table: transientBucket
        transient query: "select primaryKeyId as 'id', transientBucketId as 'alt_id', raDeg 'ra', decDeg 'dec', name 'name', sherlockClassification as 'object_classification'
            from transientBucket where object_classification is null
        transient primary id column: primaryKeyId
        transient classification column: sherlockClassification
        tunnel: False

By setting update=True the classifier will update the sherlockClassification column of the transient table with new classification and populate the sherlock_crossmatches table with key details of the crossmatched sources from the catalogues database. By setting update=False results are printed to stdout but the database is not updated (useful for dry runs and testing new algorithms),

- update key arguments values and definitions with defaults
- update return values and definitions
- update usage examples and text
- update docstring text
- check sublime snippet exists
- clip any useful text to docs mindmap
- regenerate the docs and check redendering of this docstring
classify()[source]

classify the transients selected from the transient selection query in the settings file or passed in via the CL or other code

Return

  • crossmatches – list of dictionaries of crossmatched associated sources

  • classifications – the classifications assigned to the transients post-crossmatches (dictionary of rank ordered list of classifications)

See class docstring for usage.

- update key arguments values and definitions with defaults
- update return values and definitions
- update usage examples and text
- update docstring text
- check sublime snippet exists
- clip any useful text to docs mindmap
- regenerate the docs and check redendering of this docstring
_get_transient_metadata_from_database_list()[source]

use the transient query in the settings file to generate a list of transients to corssmatch and classify

Return

- ``transientsMetadataList``
- update key arguments values and definitions with defaults
- update return values and definitions
- update usage examples and text
- update docstring text
- check sublime snippet exists
- clip any useful text to docs mindmap
- regenerate the docs and check redendering of this docstring
_update_ned_stream(transientsMetadataList)[source]

update the NED stream within the catalogues database at the locations of the transients

Key Arguments

  • transientsMetadataList – the list of transient metadata lifted from the database.

- update key arguments values and definitions with defaults
- update return values and definitions
- update usage examples and text
- update docstring text
- check sublime snippet exists
- clip any useful text to docs mindmap
- regenerate the docs and check redendering of this docstring
_remove_previous_ned_queries(coordinateList)[source]

iterate through the transient locations to see if we have recent local NED coverage of that area already

Key Arguments

  • coordinateList – set of coordinate to check for previous queries

Return

  • updatedCoordinateList – coordinate list with previous queries removed

- update key arguments values and definitions with defaults
- update return values and definitions
- update usage examples and text
- update docstring text
- check sublime snippet exists
- clip any useful text to docs mindmap
- regenerate the docs and check redendering of this docstring
_update_transient_database(crossmatches, classifications, transientsMetadataList, colMaps)[source]

update transient database with classifications and crossmatch results

Key Arguments

  • crossmatches – the crossmatches and associations resulting from the catlaogue crossmatches

  • classifications – the classifications assigned to the transients post-crossmatches (dictionary of rank ordered list of classifications)

  • transientsMetadataList – the list of transient metadata lifted from the database.

  • colMaps – maps of the important column names for each table/view in the crossmatch-catalogues database

- update key arguments values and definitions with defaults
- update return values and definitions
- update usage examples and text
- update docstring text
- check sublime snippet exists
- clip any useful text to docs mindmap
- regenerate the docs and check redendering of this docstring
_rank_classifications(crossmatchArray, colMaps)[source]

rank the classifications returned from the catalogue crossmatcher, annotate the results with a classification rank-number (most likely = 1) and a rank-score (weight of classification)

Key Arguments

  • crossmatchArrayIndex – the index of list of unranked crossmatch classifications

  • colMaps – dictionary of dictionaries with the name of the database-view (e.g. tcs_view_agn_milliquas_v4_5) as the key and the column-name dictary map as value ({view_name: {columnMap}}).

Return

  • classifications – the classifications assigned to the transients post-crossmatches

  • crossmatches – the crossmatches annotated with rankings and rank-scores

- update key arguments values and definitions with defaults
- update return values and definitions
- update usage examples and text
- update docstring text
- check sublime snippet exists
- clip any useful text to docs mindmap
- regenerate the docs and check redendering of this docstring
_print_results_to_stdout(classifications, crossmatches)[source]

print the classification and crossmatch results for a single transient object to stdout

Key Arguments

  • crossmatches – the unranked crossmatch classifications

  • classifications – the classifications assigned to the transients post-crossmatches (dictionary of rank ordered list of classifications)

- update key arguments values and definitions with defaults
- update return values and definitions
- update usage examples and text
- update docstring text
- check sublime snippet exists
- clip any useful text to docs mindmap
- regenerate the docs and check redendering of this docstring
_lighten_return(crossmatches)[source]

lighten the classification and crossmatch results for smaller database footprint

Key Arguments

  • classifications – the classifications assigned to the transients post-crossmatches (dictionary of rank ordered list of classifications)

_consolidate_coordinateList(coordinateList)[source]

match the coordinate list against itself with the parameters of the NED search queries to minimise duplicated NED queries

Key Arguments

  • coordinateList – the original coordinateList.

Return

  • updatedCoordinateList – the coordinate list with duplicated search areas removed

Usage

Todo

  • add usage info

  • create a sublime snippet for usage

  • update package tutorial if needed

usage code
- update key arguments values and definitions with defaults
- update return values and definitions
- update usage examples and text
- update docstring text
- check sublime snippet exists
- clip any useful text to docs mindmap
- regenerate the docs and check redendering of this docstring
classification_annotations()[source]

add a detialed classification annotation to each classification in the sherlock_classifications table

Key Arguments

=

Return

  • None

Usage

Todo

  • add usage info

  • create a sublime snippet for usage

  • write a command-line tool for this method

  • update package tutorial with command-line tool info if needed

usage code
- update key arguments values and definitions with defaults
- update return values and definitions
- update usage examples and text
- update docstring text
- check sublime snippet exists
- clip any useful text to docs mindmap
- regenerate the docs and check redendering of this docstring
update_classification_annotations_and_summaries(updatePeakMagnitudes=True, cl=False, crossmatches=False, classifications=False)[source]

update classification annotations and summaries

Key Arguments

  • updatePeakMagnitudes – update the peak magnitudes in the annotations to give absolute magnitudes. Default True

  • cl – reporting only to the command-line, do not update database. Default False

  • crossmatches – crossmatches will be passed for the single classifications to report annotations from command-line

  • classifications – classifications will be passed for the single classifications to have annotation appended to the dictionary for stand-alone non-database scripts

Return

  • None

Usage

Todo

  • add usage info

  • create a sublime snippet for usage

  • write a command-line tool for this method

  • update package tutorial with command-line tool info if needed

usage code
- update key arguments values and definitions with defaults
- update return values and definitions
- update usage examples and text
- update docstring text
- check sublime snippet exists
- clip any useful text to docs mindmap
- regenerate the docs and check redendering of this docstring
update_peak_magnitudes()[source]

update peak magnitudes

Key Arguments

=

Return

  • None

Usage

Todo

  • add usage info

  • create a sublime snippet for usage

  • write a command-line tool for this method

  • update package tutorial with command-line tool info if needed

usage code
- update key arguments values and definitions with defaults
- update return values and definitions
- update usage examples and text
- update docstring text
- check sublime snippet exists
- clip any useful text to docs mindmap
- regenerate the docs and check redendering of this docstring
_create_tables_if_not_exist()[source]

create the sherlock helper tables if they don’t yet exist

Key Arguments

=

Return

  • None

Usage

Todo

  • add usage info

  • create a sublime snippet for usage

  • write a command-line tool for this method

  • update package tutorial with command-line tool info if needed

usage code
- update key arguments values and definitions with defaults
- update return values and definitions
- update usage examples and text
- update docstring text
- check sublime snippet exists
- clip any useful text to docs mindmap
- regenerate the docs and check redendering of this docstring
generate_match_annotation(match, updatePeakMagnitudes=False)[source]

generate a human readale annotation for the transient-catalogue source match

Key Arguments

  • match – the source crossmatched against the transient

  • updatePeakMagnitudes – update the peak magnitudes in the annotations to give absolute magnitudes. Default False

Return

  • None

Usage

usage code

Todo

  • add usage info

  • create a sublime snippet for usage

  • write a command-line tool for this method

  • update package tutorial with command-line tool info if needed

_crossmatch_transients_against_catalogues(transientsMetadataListIndex, log, settings, colMaps)[source]

run the transients through the crossmatch algorithm in the settings file

Key Arguments

  • transientsMetadataListIndex – the list of transient metadata lifted from the database.

  • colMaps – dictionary of dictionaries with the name of the database-view (e.g. tcs_view_agn_milliquas_v4_5) as the key and the column-name dictary map as value ({view_name: {columnMap}}).

Return

  • crossmatches – a list of dictionaries of the associated sources crossmatched from the catalogues database

- update key arguments values and definitions with defaults
- update return values and definitions
- update usage examples and text
- update docstring text
- check sublime snippet exists
- clip any useful text to docs mindmap
- regenerate the docs and check redendering of this docstring