Inherits org.sleuthkit.autopsy.ingest.FileIngestModule.

Classes
class	Indexer

enum	IngestStatus

enum	StringsExtractOptions

enum	UpdateFrequency

Public Member Functions
ProcessResult	process (AbstractFile abstractFile)

void	shutDown ()

void	startUp (IngestJobContext context) throws IngestModuleException

Private Member Functions
void	cleanup ()

void	postIndexSummary ()

Static Private Member Functions
static void	putIngestStatus (long ingestJobId, long fileId, IngestStatus status)

Private Attributes
IngestJobContext	context

FileTypeDetector	fileTypeDetector

Indexer	indexer

Ingester	ingester = null

boolean	initialized = false

int	instanceNum = 0

long	jobId

final IngestServices	services = IngestServices.getInstance()

final KeywordSearchJobSettings	settings

boolean	startedSearching = false

Lookup	stringsExtractionContext

Static Private Attributes
static final List< String >	ARCHIVE_MIME_TYPES

static final Map< Long, Map< Long, IngestStatus > >	ingestStatus = new HashMap<>()

static final AtomicInteger	instanceCount = new AtomicInteger(0)

static final Logger	logger = Logger.getLogger(KeywordSearchIngestModule.class.getName())

static final List< String >	METADATA_DATE_TYPES

static final Map< String, BlackboardAttribute.ATTRIBUTE_TYPE >	METADATA_TYPES_MAP

static final IngestModuleReferenceCounter	refCounter = new IngestModuleReferenceCounter()

Detailed Description

An ingest module on a file level Performs indexing of allocated and Solr supported files, string extraction and indexing of unallocated and not Solr supported files Index commit is done periodically (determined by user set ingest update interval) Runs a periodic keyword / regular expression search on currently configured lists for ingest and writes results to blackboard Reports interesting events to Inbox and to viewers

Definition at line 88 of file KeywordSearchIngestModule.java.

Member Function Documentation

void org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.cleanup ( )

private

Common cleanup code when module stops or final searcher completes

Definition at line 425 of file KeywordSearchIngestModule.java.

void org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.postIndexSummary ( )

private

Posts inbox message with summary of text_ingested files

Definition at line 433 of file KeywordSearchIngestModule.java.

References org.sleuthkit.autopsy.ingest.IngestMessage.createMessage(), org.sleuthkit.autopsy.coreutils.MessageNotifyUtil.Notify.error(), org.sleuthkit.autopsy.ingest.IngestMessage.MessageType.INFO, org.sleuthkit.autopsy.ingest.IngestServices.postMessage(), and org.sleuthkit.autopsy.coreutils.MessageNotifyUtil.Notify.warn().

ProcessResult org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.process ( AbstractFile file )

Processes a file. Called between calls to startUp() and shutDown(). Will be called for each file in a data source.

Parameters

file	The file to analyze.

Returns: A result code indicating success or failure of the processing.

Implements org.sleuthkit.autopsy.ingest.FileIngestModule.

Definition at line 340 of file KeywordSearchIngestModule.java.

References org.sleuthkit.autopsy.ingest.IngestJobContext.fileIngestIsCancelled(), org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.Indexer.indexFile(), org.sleuthkit.autopsy.ingest.IngestModule.ProcessResult.OK, and org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.IngestStatus.SKIPPED_ERROR_INDEXING.

static void org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.putIngestStatus	(	long	ingestJobId,
		long	fileId,
		IngestStatus	status
	)

staticprivate

Records the ingest status for a given file for a given ingest job. Used for final statistics at the end of the job.

Parameters

ingestJobId	id of ingest job
fileId	id of file
status	ingest status of the file

Definition at line 212 of file KeywordSearchIngestModule.java.

void org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.shutDown ( )

After all files are ingested, execute final index commit and final search Cleanup resources, threads, timers

Implements org.sleuthkit.autopsy.ingest.FileIngestModule.

Definition at line 386 of file KeywordSearchIngestModule.java.

References org.sleuthkit.autopsy.ingest.IngestModuleReferenceCounter.decrementAndGet(), org.sleuthkit.autopsy.ingest.IngestJobContext.fileIngestIsCancelled(), org.sleuthkit.autopsy.keywordsearch.KeywordSearch.getServer(), org.sleuthkit.autopsy.keywordsearch.Server.queryNumIndexedChunks(), and org.sleuthkit.autopsy.keywordsearch.Server.queryNumIndexedFiles().

void org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.startUp ( IngestJobContext context ) throws IngestModuleException

Initializes the module for new ingest run Sets up threads, timers, retrieves settings, keyword lists to run on

Implements org.sleuthkit.autopsy.ingest.IngestModule.

Definition at line 241 of file KeywordSearchIngestModule.java.

Member Data Documentation

final List<String> org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.ARCHIVE_MIME_TYPES

staticprivate

generally text extractors should ignore archives and let unpacking modules take care of them

Definition at line 95 of file KeywordSearchIngestModule.java.

IngestJobContext org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.context

private

Definition at line 191 of file KeywordSearchIngestModule.java.

Referenced by org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.Indexer.extractStringsAndIndex().

FileTypeDetector org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.fileTypeDetector

private

Definition at line 179 of file KeywordSearchIngestModule.java.

Indexer org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.indexer

private

Definition at line 178 of file KeywordSearchIngestModule.java.

Ingester org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.ingester = null

private

Definition at line 177 of file KeywordSearchIngestModule.java.

final Map<Long, Map<Long, IngestStatus> > org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.ingestStatus = new HashMap<>()

staticprivate

Definition at line 202 of file KeywordSearchIngestModule.java.

boolean org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.initialized = false

private

Definition at line 186 of file KeywordSearchIngestModule.java.

final AtomicInteger org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.instanceCount = new AtomicInteger(0)

staticprivate

Definition at line 188 of file KeywordSearchIngestModule.java.

int org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.instanceNum = 0

private

Definition at line 189 of file KeywordSearchIngestModule.java.

long org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.jobId

private

Definition at line 187 of file KeywordSearchIngestModule.java.

final Logger org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.logger = Logger.getLogger(KeywordSearchIngestModule.class.getName())

staticprivate

Definition at line 175 of file KeywordSearchIngestModule.java.

final List<String> org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.METADATA_DATE_TYPES

staticprivate

Initial value:

= ImmutableList.of(
                    "Last-Save-Date", 
                    "Last-Printed", 
                    "Creation-Date")

Definition at line 130 of file KeywordSearchIngestModule.java.

final Map<String, BlackboardAttribute.ATTRIBUTE_TYPE> org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.METADATA_TYPES_MAP

staticprivate

Initial value:

= ImmutableMap.<String, BlackboardAttribute.ATTRIBUTE_TYPE>builder()  
                    .put("Last-Save-Date", BlackboardAttribute.ATTRIBUTE_TYPE.TSK_DATETIME_MODIFIED) 
                    .put("Last-Author", BlackboardAttribute.ATTRIBUTE_TYPE.TSK_USER_ID) 
                    .put("Creation-Date", BlackboardAttribute.ATTRIBUTE_TYPE.TSK_DATETIME_CREATED) 
                    .put("Company", BlackboardAttribute.ATTRIBUTE_TYPE.TSK_ORGANIZATION) 
                    .put("Author", BlackboardAttribute.ATTRIBUTE_TYPE.TSK_OWNER)
                    .put("Application-Name", BlackboardAttribute.ATTRIBUTE_TYPE.TSK_PROG_NAME)
                    .put("Last-Printed", BlackboardAttribute.ATTRIBUTE_TYPE.TSK_LAST_PRINTED_DATETIME)
                    .put("Producer", BlackboardAttribute.ATTRIBUTE_TYPE.TSK_PROG_NAME) 
                    .put("Title", BlackboardAttribute.ATTRIBUTE_TYPE.TSK_DESCRIPTION) 
                    .put("pdf:PDFVersion", BlackboardAttribute.ATTRIBUTE_TYPE.TSK_VERSION)
                   .build()

Definition at line 135 of file KeywordSearchIngestModule.java.

final IngestModuleReferenceCounter org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.refCounter = new IngestModuleReferenceCounter()

staticprivate

Definition at line 190 of file KeywordSearchIngestModule.java.

final IngestServices org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.services = IngestServices.getInstance()

private

Definition at line 176 of file KeywordSearchIngestModule.java.

final KeywordSearchJobSettings org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.settings

private

Definition at line 185 of file KeywordSearchIngestModule.java.

boolean org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.startedSearching = false

private

Definition at line 183 of file KeywordSearchIngestModule.java.

Lookup org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.stringsExtractionContext

private

Definition at line 184 of file KeywordSearchIngestModule.java.

The documentation for this class was generated from the following file:

/home/carriersleuth/repos/autopsy/KeywordSearch/src/org/sleuthkit/autopsy/keywordsearch/KeywordSearchIngestModule.java

Classes

Public Member Functions

Private Member Functions

Static Private Member Functions

Private Attributes

Static Private Attributes

Detailed Description

Member Function Documentation

Member Data Documentation