org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule Class Reference

Inherits org.sleuthkit.autopsy.ingest.FileIngestModule.

Classes
class	FindInHashsetsResult
class	IngestJobTotals

Public Member Functions
ProcessResult	process (AbstractFile file)
void	shutDown ()
void	startUp (org.sleuthkit.autopsy.ingest.IngestJobContext context) throws IngestModuleException

Private Member Functions
void	calculateHashes (AbstractFile file, IngestJobTotals totals) throws TskCoreException
boolean	createArtifactIfNotExists (String hashSetName, AbstractFile file, String comment, HashDb db)
FindInHashsetsResult	findInHashsets (AbstractFile file, AtomicLong totalCount, AtomicLong totalLookupTime, List< HashDb > hashSets, TskData.FileKnown statusIfFound, Function< AbstractFile, String > lookupErrorMessage)
String	generateComment (HashHitInfo hashInfo)
void	initializeHashsets (List< HashDb > allHashSets)
void	postHashSetHitToBlackboard (AbstractFile abstractFile, String md5Hash, String hashSetName, String comment, boolean showInboxMessage)
void	reportLookupError (TskException ex, AbstractFile file, Function< AbstractFile, String > lookupErrorMessage)
boolean	shouldSkip (AbstractFile file)

Static Private Member Functions
static synchronized IngestJobTotals	getTotalsForIngestJobs (long ingestJobId)
static synchronized void	postSummary (long jobId, List< HashDb > knownBadHashSets, List< HashDb > noChangeHashSets, List< HashDb > knownHashSets)

Private Attributes
Blackboard	blackboard
final HashDbManager	hashDbManager = HashDbManager.getInstance()
long	jobId
final List< HashDb >	knownBadHashSets = new ArrayList<>()
final Function< AbstractFile, String >	knownBadLookupError = (file) -> Bundle.HashDbIngestModule_lookingUpKnownBadHashValueErr(file.getName())
final List< HashDb >	knownHashSets = new ArrayList<>()
final Function< AbstractFile, String >	knownLookupError = (file) -> Bundle.HashDbIngestModule_lookingUpKnownHashValueErr(file.getName())
final List< HashDb >	noChangeHashSets = new ArrayList<>()
final Function< AbstractFile, String >	noChangeLookupError = (file) -> Bundle.HashDbIngestModule_lookingUpNoChangeHashValueErr(file.getName())
final IngestServices	services = IngestServices.getInstance()
final HashLookupModuleSettings	settings
final SleuthkitCase	skCase

Static Private Attributes
static final Logger	logger = Logger.getLogger(HashDbIngestModule.class.getName())
static final int	MAX_COMMENT_SIZE = 500
static final IngestModuleReferenceCounter	refCounter = new IngestModuleReferenceCounter()
static final HashMap< Long, IngestJobTotals >	totalsForIngestJobs = new HashMap<>()

Detailed Description

File ingest module to mark files based on hash values.

Definition at line 67 of file HashDbIngestModule.java.

Member Function Documentation

void org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.calculateHashes ( AbstractFile  file, IngestJobTotals  totals  ) throws TskCoreException

private

Generates hashes for the given file if they haven't already been set. Hashes are saved to the AbstractFile object.

Parameters

file	The file in order to determine the hash.
totals	The timing metrics for this process.

Definition at line 467 of file HashDbIngestModule.java.

References org.sleuthkit.autopsy.healthmonitor.HealthMonitor.getTimingMetric(), org.sleuthkit.autopsy.healthmonitor.HealthMonitor.submitNormalizedTimingMetric(), and org.sleuthkit.autopsy.healthmonitor.HealthMonitor.submitTimingMetric().

boolean org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.createArtifactIfNotExists ( String  hashSetName, AbstractFile  file, String  comment, HashDb  db  )

private

Creates a BlackboardArtifact if artifact does not already exist.

Parameters

hashSetName	The name of the hashset found.
file	The file that had a hash hit.
comment	The comment to associate with this artifact.
db	the database in which this file was found.

Returns

True if the operation occurred successfully and without error.

Definition at line 436 of file HashDbIngestModule.java.

References org.sleuthkit.autopsy.ingest.IngestMessage.createErrorMessage(), org.sleuthkit.autopsy.modules.hashdatabase.HashLookupModuleFactory.getModuleName(), org.sleuthkit.autopsy.modules.hashdatabase.HashDbManager.HashDb.getSendIngestMessages(), and org.sleuthkit.autopsy.ingest.IngestServices.postMessage().

FindInHashsetsResult org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.findInHashsets ( AbstractFile  file, AtomicLong  totalCount, AtomicLong  totalLookupTime, List< HashDb >  hashSets, TskData.FileKnown  statusIfFound, Function< AbstractFile, String >  lookupErrorMessage  )

private

Attempts to find an abstract file in a list of HashDB objects.

Parameters

file	The file to find.
totalCount	The total cound of files found in this type
totalLookupTime	The counter tracking the total amount of run time for this operation.
hashSets	The HashDB objects to cycle through looking for a hash hit.
statusIfFound	The FileKnown status to set on the file if the file is found in the hashSets.
lookupErrorMessage	The function that generates a message should there be an error in looking up the file in the hashSets.

Returns

Whether or not the file was found and whether or not there was an error during the operation.

Definition at line 370 of file HashDbIngestModule.java.

String org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.generateComment ( HashHitInfo  hashInfo )

private

Generates a formatted comment.

Parameters

hashInfo

The HashHitInfo.

Returns

The formatted comment.

Definition at line 409 of file HashDbIngestModule.java.

static synchronized IngestJobTotals org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.getTotalsForIngestJobs ( long  ingestJobId )

staticprivate

Definition at line 104 of file HashDbIngestModule.java.

void org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.initializeHashsets ( List< HashDb >  allHashSets )

private

Cycle through list of hashsets and place each HashDB in the appropriate list based on KnownFilesType.

Parameters

allHashSets

List of all hashsets from DB manager

Definition at line 163 of file HashDbIngestModule.java.

void org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.postHashSetHitToBlackboard ( AbstractFile  abstractFile, String  md5Hash, String  hashSetName, String  comment, boolean  showInboxMessage  )

private

Post a hash set hit to the blackboard.

Parameters

abstractFile	The file to be processed.
md5Hash	The MD5 hash value of the file.
hashSetName	The name of the hash set with which to associate the hit.
comment	A comment to be attached to the artifact.
showInboxMessage	Show a message in the inbox?

Definition at line 516 of file HashDbIngestModule.java.

References org.sleuthkit.autopsy.ingest.IngestMessage.createDataMessage(), org.sleuthkit.autopsy.coreutils.MessageNotifyUtil.Notify.error(), org.sleuthkit.autopsy.modules.hashdatabase.HashLookupModuleFactory.getModuleName(), and org.sleuthkit.autopsy.ingest.IngestServices.postMessage().

static synchronized void org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.postSummary ( long  jobId, List< HashDb >  knownBadHashSets, List< HashDb >  noChangeHashSets, List< HashDb >  knownHashSets  )

staticprivate

Post a message summarizing the results of the ingest.

Parameters

jobId	The ID of the job.
knownBadHashSets	The list of hash sets for "known bad" files.
noChangeHashSets	The list of "no change" hash sets.
knownHashSets	The list of hash sets for "known" files.

Definition at line 591 of file HashDbIngestModule.java.

References org.sleuthkit.autopsy.ingest.IngestMessage.createMessage(), org.sleuthkit.autopsy.ingest.IngestServices.getInstance(), org.sleuthkit.autopsy.modules.hashdatabase.HashLookupModuleFactory.getModuleName(), org.sleuthkit.autopsy.ingest.IngestMessage.MessageType.INFO, org.sleuthkit.autopsy.ingest.IngestServices.postMessage(), org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.IngestJobTotals.totalCalctime, org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.IngestJobTotals.totalKnownBadCount, org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.IngestJobTotals.totalLookuptime, and org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.IngestJobTotals.totalNoChangeCount.

ProcessResult org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.process

(

AbstractFile

file

)

Processes a file. Called between calls to startUp() and shutDown(). Will be called for each file in a data source.

Parameters

file	The file to analyze.

Returns

A result code indicating success or failure of the processing.

Implements org.sleuthkit.autopsy.ingest.FileIngestModule.

Definition at line 196 of file HashDbIngestModule.java.

References org.sleuthkit.autopsy.ingest.IngestMessage.createErrorMessage(), org.sleuthkit.autopsy.ingest.IngestModule.ProcessResult.ERROR, org.sleuthkit.autopsy.casemodule.Case.getCurrentCaseThrows(), org.sleuthkit.autopsy.modules.hashdatabase.HashLookupModuleFactory.getModuleName(), org.sleuthkit.autopsy.casemodule.Case.getSleuthkitCase(), org.sleuthkit.autopsy.ingest.IngestModule.ProcessResult.OK, org.sleuthkit.autopsy.ingest.IngestServices.postMessage(), org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.IngestJobTotals.totalKnownBadCount, org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.IngestJobTotals.totalLookuptime, and org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.IngestJobTotals.totalNoChangeCount.

void org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.reportLookupError ( TskException  ex, AbstractFile  file, Function< AbstractFile, String >  lookupErrorMessage  )

private

Reports an error when an issue is encountered looking up a file.

Parameters

ex	The exception thrown in the error.
file	The file for which this error applies.
lookupErrorMessage	The function that generates an error message specific to which piece of the ingest processing failed.

Definition at line 309 of file HashDbIngestModule.java.

References org.sleuthkit.autopsy.ingest.IngestMessage.createErrorMessage(), org.sleuthkit.autopsy.modules.hashdatabase.HashLookupModuleFactory.getModuleName(), and org.sleuthkit.autopsy.ingest.IngestServices.postMessage().

boolean org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.shouldSkip ( AbstractFile  file )

private

Returns true if this file should be skipped for processing.

Parameters

file	The file to potentially skip.

Returns

True if this file should be skipped.

Definition at line 275 of file HashDbIngestModule.java.

void org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.shutDown

(

)

Invoked by Autopsy when an ingest job is completed (either because the data has been analyzed or because the job was canceled - check IngestJobContext.fileIngestIsCancelled()), before the ingest module instance is discarded. The module should respond by doing things like releasing private resources, submitting final results, and posting a final ingest message.

Implements org.sleuthkit.autopsy.ingest.FileIngestModule.

Definition at line 631 of file HashDbIngestModule.java.

References org.sleuthkit.autopsy.ingest.IngestModuleReferenceCounter.decrementAndGet().

void org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.startUp

(

org.sleuthkit.autopsy.ingest.IngestJobContext

context

)

throws IngestModuleException

Invoked by Autopsy to allow an ingest module instance to set up any internal data structures and acquire any private resources it will need during an ingest job. If the module depends on loading any resources, it should do so in this method so that it can throw an exception in the case of an error and alert the user. Exceptions that are thrown from process() and shutDown() are logged, but do not stop processing of the data source.

Parameters

context

Provides data and services specific to the ingest job and the ingest pipeline of which the module is a part.

Exceptions

org.sleuthkit.autopsy.ingest.IngestModule.IngestModuleException

Implements org.sleuthkit.autopsy.ingest.IngestModule.

Definition at line 128 of file HashDbIngestModule.java.

References org.sleuthkit.autopsy.ingest.IngestMessage.createWarningMessage(), org.sleuthkit.autopsy.modules.hashdatabase.HashDbManager.getAllHashSets(), org.sleuthkit.autopsy.modules.hashdatabase.HashLookupModuleFactory.getModuleName(), org.sleuthkit.autopsy.ingest.IngestModuleReferenceCounter.incrementAndGet(), and org.sleuthkit.autopsy.ingest.IngestServices.postMessage().

Member Data Documentation

Blackboard org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.blackboard

private

Definition at line 91 of file HashDbIngestModule.java.

final HashDbManager org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.hashDbManager = HashDbManager.getInstance()

private

Definition at line 83 of file HashDbIngestModule.java.

long org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.jobId

private

Definition at line 88 of file HashDbIngestModule.java.

final List<HashDb> org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.knownBadHashSets = new ArrayList<>()

private

Definition at line 85 of file HashDbIngestModule.java.

final Function<AbstractFile, String> org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.knownBadLookupError = (file) -> Bundle.HashDbIngestModule_lookingUpKnownBadHashValueErr(file.getName())

private

Definition at line 72 of file HashDbIngestModule.java.

final List<HashDb> org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.knownHashSets = new ArrayList<>()

private

Definition at line 86 of file HashDbIngestModule.java.

final Function<AbstractFile, String> org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.knownLookupError = (file) -> Bundle.HashDbIngestModule_lookingUpKnownHashValueErr(file.getName())

private

Definition at line 78 of file HashDbIngestModule.java.

final Logger org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.logger = Logger.getLogger(HashDbIngestModule.class.getName())

staticprivate

Definition at line 69 of file HashDbIngestModule.java.

final int org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.MAX_COMMENT_SIZE = 500

staticprivate

Definition at line 80 of file HashDbIngestModule.java.

final List<HashDb> org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.noChangeHashSets = new ArrayList<>()

private

Definition at line 87 of file HashDbIngestModule.java.

final Function<AbstractFile, String> org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.noChangeLookupError = (file) -> Bundle.HashDbIngestModule_lookingUpNoChangeHashValueErr(file.getName())

private

Definition at line 75 of file HashDbIngestModule.java.

final IngestModuleReferenceCounter org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.refCounter = new IngestModuleReferenceCounter()

staticprivate

Definition at line 90 of file HashDbIngestModule.java.

final IngestServices org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.services = IngestServices.getInstance()

private

Definition at line 81 of file HashDbIngestModule.java.

final HashLookupModuleSettings org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.settings

private

Definition at line 84 of file HashDbIngestModule.java.

final SleuthkitCase org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.skCase

private

Definition at line 82 of file HashDbIngestModule.java.

final HashMap<Long, IngestJobTotals> org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.totalsForIngestJobs = new HashMap<>()

staticprivate

Definition at line 89 of file HashDbIngestModule.java.

---

The documentation for this class was generated from the following file:

• /home/carriersleuth/repos/autopsy/Core/src/org/sleuthkit/autopsy/modules/hashdatabase/HashDbIngestModule.java