org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule Class Reference

Inherits org.sleuthkit.autopsy.ingest.FileIngestModule.

Classes
class	Indexer
enum	IngestStatus
enum	UpdateFrequency

Public Member Functions
ProcessResult	process (AbstractFile abstractFile)
void	shutDown ()
void	startUp (IngestJobContext context) throws IngestModuleException

Private Member Functions
void	cleanup ()
void	postIndexSummary ()
void	stop ()

Static Private Member Functions
static void	putIngestStatus (long ingestJobId, long fileId, IngestStatus status)

Private Attributes
IngestJobContext	context
long	dataSourceId
FileTypeDetector	fileTypeDetector
Indexer	indexer
Ingester	ingester = null
boolean	initialized = false
int	instanceNum = 0
long	jobId
final IngestServices	services = IngestServices.getInstance()
final KeywordSearchJobSettings	settings
boolean	startedSearching = false
StringsTextExtractor	stringExtractor
List< TextExtractor >	textExtractors

Static Private Attributes
static final Map< Long, Map< Long, IngestStatus > >	ingestStatus = new HashMap<>()
static final AtomicInteger	instanceCount = new AtomicInteger(0)
static final Logger	logger = Logger.getLogger(KeywordSearchIngestModule.class.getName())
static final IngestModuleReferenceCounter	refCounter = new IngestModuleReferenceCounter()

Detailed Description

An ingest module on a file level Performs indexing of allocated and Solr supported files, string extraction and indexing of unallocated and not Solr supported files Index commit is done periodically (determined by user set ingest update interval) Runs a periodic keyword / regular expression search on currently configured lists for ingest and writes results to blackboard Reports interesting events to Inbox and to viewers

Definition at line 56 of file KeywordSearchIngestModule.java.

Member Function Documentation

void org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.cleanup ( )

private

Common cleanup code when module stops or final searcher completes

Definition at line 330 of file KeywordSearchIngestModule.java.

Referenced by org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.shutDown(), and org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.stop().

void org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.postIndexSummary ( )

private

Posts inbox message with summary of text_ingested files

Definition at line 341 of file KeywordSearchIngestModule.java.

References org.sleuthkit.autopsy.ingest.IngestMessage.createMessage(), org.sleuthkit.autopsy.coreutils.MessageNotifyUtil.Notify.error(), org.sleuthkit.autopsy.ingest.IngestMessage.MessageType.INFO, org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.ingestStatus, org.sleuthkit.autopsy.ingest.IngestServices.postMessage(), and org.sleuthkit.autopsy.coreutils.MessageNotifyUtil.Notify.warn().

Referenced by org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.shutDown().

ProcessResult org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.process

(

AbstractFile

file

)

Processes a file. Called between calls to startUp() and shutDown(). Will be called for each file in a data source.

Parameters

file	The file to analyze.

Returns

A result code indicating success or failure of the processing.

Implements org.sleuthkit.autopsy.ingest.FileIngestModule.

Definition at line 242 of file KeywordSearchIngestModule.java.

References org.sleuthkit.autopsy.keywordsearch.SearchRunner.getInstance(), org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.Indexer.indexFile(), org.sleuthkit.autopsy.ingest.IngestModule.ProcessResult.OK, org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.putIngestStatus(), org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.IngestStatus.SKIPPED_ERROR_INDEXING, and org.sleuthkit.autopsy.keywordsearch.SearchRunner.startJob().

static void org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.putIngestStatus ( long  ingestJobId, long  fileId, IngestStatus  status  )

staticprivate

Definition at line 107 of file KeywordSearchIngestModule.java.

References org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.ingestStatus.

Referenced by org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.Indexer.extractStringsAndIndex(), org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.Indexer.indexFile(), and org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.process().

void org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.shutDown

(

)

After all files are ingested, execute final index commit and final search Cleanup resources, threads, timers

Implements org.sleuthkit.autopsy.ingest.FileIngestModule.

Definition at line 279 of file KeywordSearchIngestModule.java.

References org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.cleanup(), org.sleuthkit.autopsy.ingest.IngestModuleReferenceCounter.decrementAndGet(), org.sleuthkit.autopsy.keywordsearch.SearchRunner.endJob(), org.sleuthkit.autopsy.ingest.IngestJobContext.fileIngestIsCancelled(), org.sleuthkit.autopsy.keywordsearch.SearchRunner.getInstance(), org.sleuthkit.autopsy.keywordsearch.KeywordSearch.getServer(), org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.ingestStatus, org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.postIndexSummary(), org.sleuthkit.autopsy.keywordsearch.Server.queryNumIndexedChunks(), org.sleuthkit.autopsy.keywordsearch.Server.queryNumIndexedFiles(), and org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.stop().

void org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.startUp

(

IngestJobContext

context

)

throws IngestModuleException

Initializes the module for new ingest run Sets up threads, timers, retrieves settings, keyword lists to run on

Implements org.sleuthkit.autopsy.ingest.IngestModule.

Definition at line 131 of file KeywordSearchIngestModule.java.

References org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.context, org.sleuthkit.autopsy.ingest.IngestMessage.createErrorMessage(), org.sleuthkit.autopsy.ingest.IngestMessage.createWarningMessage(), org.sleuthkit.autopsy.casemodule.Case.getCaseType(), org.sleuthkit.autopsy.casemodule.Case.getCurrentCase(), org.sleuthkit.autopsy.ingest.IngestJobContext.getDataSource(), org.sleuthkit.autopsy.core.UserPreferences.getIndexingServerHost(), org.sleuthkit.autopsy.core.UserPreferences.getIndexingServerPort(), org.sleuthkit.autopsy.keywordsearch.Server.getIngester(), org.sleuthkit.autopsy.ingest.IngestJobContext.getJobId(), org.sleuthkit.autopsy.keywordsearch.KeywordSearch.getServer(), org.sleuthkit.autopsy.ingest.IngestModuleReferenceCounter.incrementAndGet(), org.sleuthkit.autopsy.casemodule.Case.CaseType.MULTI_USER_CASE, org.sleuthkit.autopsy.ingest.IngestServices.postMessage(), org.sleuthkit.autopsy.keywordsearch.Server.queryNumIndexedDocuments(), and org.sleuthkit.autopsy.keywordsearchservice.KeywordSearchService.tryConnect().

void org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.stop ( )

private

Handle stop event (ingest interrupted) Cleanup resources, threads, timers

Definition at line 319 of file KeywordSearchIngestModule.java.

References org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.cleanup(), org.sleuthkit.autopsy.keywordsearch.SearchRunner.getInstance(), and org.sleuthkit.autopsy.keywordsearch.SearchRunner.stopJob().

Referenced by org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.shutDown().

Member Data Documentation

IngestJobContext org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.context

private

Definition at line 94 of file KeywordSearchIngestModule.java.

Referenced by org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.startUp().

long org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.dataSourceId

private

Definition at line 90 of file KeywordSearchIngestModule.java.

FileTypeDetector org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.fileTypeDetector

private

Definition at line 80 of file KeywordSearchIngestModule.java.

Indexer org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.indexer

private

Definition at line 79 of file KeywordSearchIngestModule.java.

Ingester org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.ingester = null

private

Definition at line 78 of file KeywordSearchIngestModule.java.

final Map<Long, Map<Long, IngestStatus> > org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.ingestStatus = new HashMap<>()

staticprivate

Definition at line 105 of file KeywordSearchIngestModule.java.

Referenced by org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.postIndexSummary(), org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.putIngestStatus(), and org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.shutDown().

boolean org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.initialized = false

private

Definition at line 88 of file KeywordSearchIngestModule.java.

final AtomicInteger org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.instanceCount = new AtomicInteger(0)

staticprivate

Definition at line 91 of file KeywordSearchIngestModule.java.

int org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.instanceNum = 0

private

Definition at line 92 of file KeywordSearchIngestModule.java.

long org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.jobId

private

Definition at line 89 of file KeywordSearchIngestModule.java.

final Logger org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.logger = Logger.getLogger(KeywordSearchIngestModule.class.getName())

staticprivate

Definition at line 76 of file KeywordSearchIngestModule.java.

final IngestModuleReferenceCounter org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.refCounter = new IngestModuleReferenceCounter()

staticprivate

Definition at line 93 of file KeywordSearchIngestModule.java.

final IngestServices org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.services = IngestServices.getInstance()

private

Definition at line 77 of file KeywordSearchIngestModule.java.

final KeywordSearchJobSettings org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.settings

private

Definition at line 87 of file KeywordSearchIngestModule.java.

boolean org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.startedSearching = false

private

Definition at line 84 of file KeywordSearchIngestModule.java.

StringsTextExtractor org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.stringExtractor

private

Definition at line 86 of file KeywordSearchIngestModule.java.

List<TextExtractor> org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.textExtractors

private

Definition at line 85 of file KeywordSearchIngestModule.java.

---

The documentation for this class was generated from the following file:

• /home/carriersleuth/repos/autopsy/KeywordSearch/src/org/sleuthkit/autopsy/keywordsearch/KeywordSearchIngestModule.java