Autopsy
4.15.0
Graphical digital forensics platform for The Sleuth Kit and other tools.
|
Classes | |
class | Snapshot |
enum | Stages |
Private Member Functions | |
void | addIngestModules (List< IngestModuleTemplate > templates, IngestModuleType type, SleuthkitCase skCase) throws TskCoreException |
void | checkForStageCompleted () |
void | createIngestPipelines () |
void | finish () |
void | finishFirstStage () |
boolean | hasFileIngestPipeline () |
boolean | hasFirstStageDataSourceIngestPipeline () |
boolean | hasSecondStageDataSourceIngestPipeline () |
void | logErrorMessage (Level level, String message, Throwable throwable) |
void | logErrorMessage (Level level, String message) |
void | logInfoMessage (String message) |
void | logIngestModuleErrors (List< IngestModuleError > errors) |
void | startDataSourceIngestProgressBar () |
void | startFileIngestProgressBar () |
void | startFirstStage () |
void | startSecondStage () |
List< IngestModuleError > | startUpIngestPipelines () |
Static Private Member Functions | |
static void | addModule (Map< String, IngestModuleTemplate > mapping, Map< String, IngestModuleTemplate > jythonMapping, IngestModuleTemplate template) |
static void | addOrdered (final List< IngestModuleTemplate > dest, final Map< String, IngestModuleTemplate > src, final Map< String, IngestModuleTemplate > jythonSrc) |
static List< IngestModuleTemplate > | getConfiguredIngestModuleTemplates (Map< String, IngestModuleTemplate > ingestModuleTemplates, Map< String, IngestModuleTemplate > jythonIngestModuleTemplates, List< String > pipelineConfig) |
static String | getJythonName (String canonicalName) |
Private Attributes | |
volatile IngestJob.CancellationReason | cancellationReason = IngestJob.CancellationReason.NOT_CANCELLED |
volatile boolean | cancelled |
final List< String > | cancelledDataSourceIngestModules = new CopyOnWriteArrayList<>() |
final long | createTime |
volatile boolean | currentDataSourceIngestModuleCancelled |
DataSourceIngestPipeline | currentDataSourceIngestPipeline |
String | currentFileIngestModule = "" |
String | currentFileIngestTask = "" |
final Content | dataSource |
final Object | dataSourceIngestPipelineLock = new Object() |
ProgressHandle | dataSourceIngestProgress |
final Object | dataSourceIngestProgressLock = new Object() |
final boolean | doUI |
long | estimatedFilesToProcess |
final List< FileIngestPipeline > | fileIngestPipelines = new ArrayList<>() |
final LinkedBlockingQueue< FileIngestPipeline > | fileIngestPipelinesQueue = new LinkedBlockingQueue<>() |
ProgressHandle | fileIngestProgress |
final Object | fileIngestProgressLock = new Object() |
final List< AbstractFile > | files = new ArrayList<>() |
final List< String > | filesInProgress = new ArrayList<>() |
DataSourceIngestPipeline | firstStageDataSourceIngestPipeline |
final long | id |
volatile IngestJobInfo | ingestJob |
final List< IngestModuleInfo > | ingestModules = new ArrayList<>() |
final IngestJob | parentJob |
long | processedFiles |
DataSourceIngestPipeline | secondStageDataSourceIngestPipeline |
final IngestJobSettings | settings |
volatile Stages | stage = DataSourceIngestJob.Stages.INITIALIZATION |
final Object | stageCompletionCheckLock = new Object() |
Static Private Attributes | |
static String | AUTOPSY_MODULE_PREFIX = "org.sleuthkit.autopsy" |
static final Pattern | JYTHON_REGEX = Pattern.compile("org\\.python\\.proxies\\.(.+?)\\$(.+?)(\\$[0-9]*)?$") |
static final Logger | logger = Logger.getLogger(DataSourceIngestJob.class.getName()) |
static final AtomicLong | nextJobId = new AtomicLong(0L) |
static final IngestTasksScheduler | taskScheduler = IngestTasksScheduler.getInstance() |
Encapsulates a data source and the ingest module pipelines used to process it.
Definition at line 62 of file DataSourceIngestJob.java.
|
private |
Definition at line 377 of file DataSourceIngestJob.java.
References org.sleuthkit.autopsy.python.FactoryClassNameNormalizer.normalize().
Referenced by org.sleuthkit.autopsy.ingest.DataSourceIngestJob.createIngestPipelines().
|
staticprivate |
Adds a template to the appropriate map. If the class is a jython class, then it is added to the jython map. Otherwise, it is added to the mapping.
mapping | Mapping for non-jython objects. |
jythonMapping | Mapping for jython objects. |
template | The template to add. |
Definition at line 283 of file DataSourceIngestJob.java.
References org.sleuthkit.autopsy.ingest.DataSourceIngestJob.getJythonName().
Referenced by org.sleuthkit.autopsy.ingest.DataSourceIngestJob.createIngestPipelines().
|
staticprivate |
Adds ingest modules to a list with autopsy modules first and third party modules next.
dest | The destination for the modules to be added. |
src | A map of fully qualified class name mapped to the IngestModuleTemplate. |
jythonSrc | A map of fully qualified class name mapped to the IngestModuleTemplate for jython modules. |
Definition at line 237 of file DataSourceIngestJob.java.
Referenced by org.sleuthkit.autopsy.ingest.DataSourceIngestJob.createIngestPipelines().
|
private |
Checks to see if the ingest tasks for the current stage of this job are completed and does a stage transition if they are.
Definition at line 718 of file DataSourceIngestJob.java.
References org.sleuthkit.autopsy.ingest.DataSourceIngestJob.finish(), org.sleuthkit.autopsy.ingest.DataSourceIngestJob.finishFirstStage(), org.sleuthkit.autopsy.ingest.DataSourceIngestJob.stageCompletionCheckLock, and org.sleuthkit.autopsy.ingest.DataSourceIngestJob.taskScheduler.
Referenced by org.sleuthkit.autopsy.ingest.DataSourceIngestJob.startFirstStage().
|
private |
Creates the file and data source ingest pipelines.
Make mappings of ingest module factory class names to templates.
Use the mappings and the ingest pipelines configuration to create ordered lists of ingest module templates for each ingest pipeline.
Add any module templates that were not specified in the pipelines configuration to an appropriate pipeline - either the first stage data source ingest pipeline or the file ingest pipeline.
Construct the data source ingest pipelines.
Construct the file ingest pipelines, one per file ingest thread.
The current thread was interrupted while blocked on a full queue. Blocking should actually never happen here, but reset the interrupted flag rather than just swallowing the exception.
Definition at line 298 of file DataSourceIngestJob.java.
References org.sleuthkit.autopsy.ingest.DataSourceIngestJob.addIngestModules(), org.sleuthkit.autopsy.ingest.DataSourceIngestJob.addModule(), org.sleuthkit.autopsy.ingest.DataSourceIngestJob.addOrdered(), org.sleuthkit.autopsy.ingest.DataSourceIngestJob.getConfiguredIngestModuleTemplates(), org.sleuthkit.autopsy.casemodule.Case.getCurrentCaseThrows(), org.sleuthkit.autopsy.ingest.IngestJobSettings.getEnabledIngestModuleTemplates(), org.sleuthkit.autopsy.ingest.IngestManager.getInstance(), org.sleuthkit.autopsy.ingest.IngestManager.getNumberOfFileIngestThreads(), org.sleuthkit.autopsy.casemodule.Case.getSleuthkitCase(), and org.sleuthkit.autopsy.ingest.DataSourceIngestJob.logErrorMessage().
|
private |
Shuts down the ingest pipelines and progress bars for this job.
Definition at line 787 of file DataSourceIngestJob.java.
References org.sleuthkit.autopsy.ingest.DataSourceIngestJob.dataSourceIngestProgressLock, org.sleuthkit.autopsy.ingest.DataSourceIngestJob.Stages.FINALIZATION, org.sleuthkit.autopsy.ingest.DataSourceIngestJob.logErrorMessage(), and org.sleuthkit.autopsy.ingest.DataSourceIngestJob.logInfoMessage().
Referenced by org.sleuthkit.autopsy.ingest.DataSourceIngestJob.checkForStageCompleted(), and org.sleuthkit.autopsy.ingest.DataSourceIngestJob.finishFirstStage().
|
private |
Shuts down the first stage ingest pipelines and progress bars for this job and starts the second stage, if appropriate.
Start the second stage, if appropriate.
Definition at line 737 of file DataSourceIngestJob.java.
References org.sleuthkit.autopsy.ingest.DataSourceIngestJob.dataSourceIngestProgressLock, org.sleuthkit.autopsy.ingest.DataSourceIngestJob.fileIngestProgressLock, org.sleuthkit.autopsy.ingest.DataSourceIngestJob.finish(), org.sleuthkit.autopsy.ingest.DataSourceIngestJob.hasSecondStageDataSourceIngestPipeline(), org.sleuthkit.autopsy.ingest.DataSourceIngestJob.logInfoMessage(), org.sleuthkit.autopsy.ingest.DataSourceIngestJob.logIngestModuleErrors(), and org.sleuthkit.autopsy.ingest.DataSourceIngestJob.startSecondStage().
Referenced by org.sleuthkit.autopsy.ingest.DataSourceIngestJob.checkForStageCompleted().
|
staticprivate |
Uses an input collection of ingest module templates and a pipeline configuration, i.e., an ordered list of ingest module factory class names, to create an ordered output list of ingest module templates for an ingest pipeline. The ingest module templates are removed from the input collection as they are added to the output collection.
ingestModuleTemplates | A mapping of ingest module factory class names to ingest module templates. |
jythonIngestModuleTemplates | A mapping of jython processed class names to jython ingest module templates. |
pipelineConfig | An ordered list of ingest module factory class names representing an ingest pipeline. |
Definition at line 403 of file DataSourceIngestJob.java.
Referenced by org.sleuthkit.autopsy.ingest.DataSourceIngestJob.createIngestPipelines().
|
staticprivate |
Takes a classname like "org.python.proxies.GPX_Parser_Module$GPXParserFileIngestModuleFactory$14" and provides "GPX_Parser_Module.GPXParserFileIngestModuleFactory" or null if not in jython package.
canonicalName | The canonical name. |
Definition at line 265 of file DataSourceIngestJob.java.
Referenced by org.sleuthkit.autopsy.ingest.DataSourceIngestJob.addModule().
|
private |
Checks to see if this job has a file level ingest pipeline.
Definition at line 498 of file DataSourceIngestJob.java.
Referenced by org.sleuthkit.autopsy.ingest.DataSourceIngestJob.startFirstStage().
|
private |
Checks to see if this job has a first stage data source level ingest pipeline.
Definition at line 479 of file DataSourceIngestJob.java.
Referenced by org.sleuthkit.autopsy.ingest.DataSourceIngestJob.startFirstStage().
|
private |
Checks to see if this job has a second stage data source level ingest pipeline.
Definition at line 489 of file DataSourceIngestJob.java.
Referenced by org.sleuthkit.autopsy.ingest.DataSourceIngestJob.finishFirstStage().
|
private |
Writes an error message to the application log that includes the data source name, data source object id, and the job id.
level | The logging level for the message. |
message | The message. |
throwable | The throwable associated with the error. |
Definition at line 1179 of file DataSourceIngestJob.java.
References org.sleuthkit.autopsy.ingest.DataSourceIngestJob.id.
Referenced by org.sleuthkit.autopsy.ingest.DataSourceIngestJob.createIngestPipelines(), org.sleuthkit.autopsy.ingest.DataSourceIngestJob.finish(), and org.sleuthkit.autopsy.ingest.DataSourceIngestJob.logIngestModuleErrors().
|
private |
Writes an error message to the application log that includes the data source name, data source object id, and the job id.
level | The logging level for the message. |
message | The message. |
Definition at line 1190 of file DataSourceIngestJob.java.
References org.sleuthkit.autopsy.ingest.DataSourceIngestJob.id.
|
private |
Writes an info message to the application log that includes the data source name, data source object id, and the job id.
message | The message. |
Definition at line 1167 of file DataSourceIngestJob.java.
References org.sleuthkit.autopsy.ingest.DataSourceIngestJob.id.
Referenced by org.sleuthkit.autopsy.ingest.DataSourceIngestJob.finish(), org.sleuthkit.autopsy.ingest.DataSourceIngestJob.finishFirstStage(), org.sleuthkit.autopsy.ingest.DataSourceIngestJob.startFirstStage(), and org.sleuthkit.autopsy.ingest.DataSourceIngestJob.startSecondStage().
|
private |
Write ingest module errors to the log.
errors | The errors. |
Definition at line 1199 of file DataSourceIngestJob.java.
References org.sleuthkit.autopsy.ingest.DataSourceIngestJob.logErrorMessage().
Referenced by org.sleuthkit.autopsy.ingest.DataSourceIngestJob.finishFirstStage(), and org.sleuthkit.autopsy.ingest.DataSourceIngestJob.startUpIngestPipelines().
|
private |
Starts a data source level ingest progress bar for this job.
Definition at line 656 of file DataSourceIngestJob.java.
References org.sleuthkit.autopsy.ingest.DataSourceIngestJob.dataSourceIngestProgressLock, and org.sleuthkit.autopsy.ingest.IngestJob.CancellationReason.USER_CANCELLED.
Referenced by org.sleuthkit.autopsy.ingest.DataSourceIngestJob.startFirstStage(), and org.sleuthkit.autopsy.ingest.DataSourceIngestJob.startSecondStage().
|
private |
Starts the file level ingest progress bar for this job.
Definition at line 691 of file DataSourceIngestJob.java.
References org.sleuthkit.autopsy.ingest.DataSourceIngestJob.fileIngestProgressLock, and org.sleuthkit.autopsy.ingest.IngestJob.CancellationReason.USER_CANCELLED.
Referenced by org.sleuthkit.autopsy.ingest.DataSourceIngestJob.startFirstStage().
|
private |
Starts the first stage of this job.
Start one or both of the first stage ingest progress bars.
Make the first stage data source level ingest pipeline the current data source level pipeline.
Schedule the first stage tasks.
No data source ingest task has been scheduled for this stage, and it is possible, if unlikely, that no file ingest tasks were actually scheduled since there are files that get filtered out by the tasks scheduler. In this special case, an ingest thread will never get to check for completion of this stage of the job, so do it now.
Definition at line 583 of file DataSourceIngestJob.java.
References org.sleuthkit.autopsy.ingest.DataSourceIngestJob.checkForStageCompleted(), org.sleuthkit.autopsy.ingest.DataSourceIngestJob.dataSourceIngestPipelineLock, org.sleuthkit.autopsy.ingest.DataSourceIngestJob.fileIngestProgressLock, org.sleuthkit.autopsy.ingest.DataSourceIngestJob.Stages.FIRST, org.sleuthkit.autopsy.ingest.DataSourceIngestJob.firstStageDataSourceIngestPipeline, org.sleuthkit.autopsy.ingest.DataSourceIngestJob.hasFileIngestPipeline(), org.sleuthkit.autopsy.ingest.DataSourceIngestJob.hasFirstStageDataSourceIngestPipeline(), org.sleuthkit.autopsy.ingest.DataSourceIngestJob.logInfoMessage(), org.sleuthkit.autopsy.ingest.DataSourceIngestJob.startDataSourceIngestProgressBar(), org.sleuthkit.autopsy.ingest.DataSourceIngestJob.startFileIngestProgressBar(), and org.sleuthkit.autopsy.ingest.DataSourceIngestJob.taskScheduler.
|
private |
Starts the second stage of this ingest job.
Definition at line 640 of file DataSourceIngestJob.java.
References org.sleuthkit.autopsy.ingest.DataSourceIngestJob.dataSourceIngestPipelineLock, org.sleuthkit.autopsy.ingest.DataSourceIngestJob.logInfoMessage(), org.sleuthkit.autopsy.ingest.DataSourceIngestJob.Stages.SECOND, org.sleuthkit.autopsy.ingest.DataSourceIngestJob.secondStageDataSourceIngestPipeline, org.sleuthkit.autopsy.ingest.DataSourceIngestJob.startDataSourceIngestProgressBar(), and org.sleuthkit.autopsy.ingest.DataSourceIngestJob.taskScheduler.
Referenced by org.sleuthkit.autopsy.ingest.DataSourceIngestJob.finishFirstStage().
|
private |
Starts up each of the ingest pipelines for this job to collect any file and data source level ingest modules errors that might occur.
Definition at line 535 of file DataSourceIngestJob.java.
References org.sleuthkit.autopsy.ingest.DataSourceIngestJob.logIngestModuleErrors().
|
staticprivate |
Definition at line 64 of file DataSourceIngestJob.java.
|
private |
Definition at line 148 of file DataSourceIngestJob.java.
|
private |
Definition at line 147 of file DataSourceIngestJob.java.
|
private |
Definition at line 146 of file DataSourceIngestJob.java.
|
private |
A data source ingest job uses this field to report its creation time.
Definition at line 186 of file DataSourceIngestJob.java.
|
private |
A data source ingest job supports cancellation of either the currently running data source level ingest module or the entire ingest job.
TODO: The currentDataSourceIngestModuleCancelled field and all of the code concerned with it is a hack to avoid an API change. The next time an API change is legal, a cancel() method needs to be added to the IngestModule interface and this field should be removed. The "ingest job is canceled" queries should also be removed from the IngestJobContext class.
Definition at line 145 of file DataSourceIngestJob.java.
|
private |
Definition at line 122 of file DataSourceIngestJob.java.
|
private |
Definition at line 178 of file DataSourceIngestJob.java.
|
private |
Definition at line 179 of file DataSourceIngestJob.java.
|
private |
Definition at line 81 of file DataSourceIngestJob.java.
|
private |
A data source ingest job has separate data source level ingest module pipelines for the first and second processing stages. Longer running, lower priority modules belong in the second stage pipeline, although this cannot be enforced. Note that the pipelines for both stages are created at job start up to allow for verification that they both can be started up without errors.
Definition at line 119 of file DataSourceIngestJob.java.
Referenced by org.sleuthkit.autopsy.ingest.DataSourceIngestJob.startFirstStage(), and org.sleuthkit.autopsy.ingest.DataSourceIngestJob.startSecondStage().
|
private |
Definition at line 167 of file DataSourceIngestJob.java.
|
private |
A data source ingest job uses these fields to report data source level ingest progress.
Definition at line 166 of file DataSourceIngestJob.java.
Referenced by org.sleuthkit.autopsy.ingest.DataSourceIngestJob.finish(), org.sleuthkit.autopsy.ingest.DataSourceIngestJob.finishFirstStage(), and org.sleuthkit.autopsy.ingest.DataSourceIngestJob.startDataSourceIngestProgressBar().
|
private |
A data source ingest job can run interactively using NetBeans progress handles.
Definition at line 160 of file DataSourceIngestJob.java.
|
private |
Definition at line 175 of file DataSourceIngestJob.java.
|
private |
Definition at line 132 of file DataSourceIngestJob.java.
|
private |
A data source ingest job has a collection of identical file level ingest module pipelines, one for each file level ingest thread in the ingest manager. A blocking queue is used to dole out the pipelines to the threads and an ordinary list is used when the ingest job needs to access the pipelines to query their status.
Definition at line 131 of file DataSourceIngestJob.java.
|
private |
Definition at line 177 of file DataSourceIngestJob.java.
|
private |
A data source ingest job uses these fields to report file level ingest progress.
Definition at line 173 of file DataSourceIngestJob.java.
Referenced by org.sleuthkit.autopsy.ingest.DataSourceIngestJob.finishFirstStage(), org.sleuthkit.autopsy.ingest.DataSourceIngestJob.startFileIngestProgressBar(), and org.sleuthkit.autopsy.ingest.DataSourceIngestJob.startFirstStage().
|
private |
Definition at line 82 of file DataSourceIngestJob.java.
|
private |
Definition at line 174 of file DataSourceIngestJob.java.
|
private |
Definition at line 120 of file DataSourceIngestJob.java.
Referenced by org.sleuthkit.autopsy.ingest.DataSourceIngestJob.startFirstStage().
|
private |
Definition at line 79 of file DataSourceIngestJob.java.
Referenced by org.sleuthkit.autopsy.ingest.DataSourceIngestJob.logErrorMessage(), and org.sleuthkit.autopsy.ingest.DataSourceIngestJob.logInfoMessage().
|
private |
Definition at line 181 of file DataSourceIngestJob.java.
|
private |
Definition at line 180 of file DataSourceIngestJob.java.
|
staticprivate |
Definition at line 69 of file DataSourceIngestJob.java.
|
staticprivate |
Definition at line 66 of file DataSourceIngestJob.java.
|
staticprivate |
Definition at line 78 of file DataSourceIngestJob.java.
|
private |
These fields define a data source ingest job: the parent ingest job, an ID, the user's ingest job settings, and the data source to be analyzed. Optionally, there is a set of files to be analyzed instead of analyzing all of the files in the data source.
Definition at line 77 of file DataSourceIngestJob.java.
|
private |
Definition at line 176 of file DataSourceIngestJob.java.
|
private |
Definition at line 121 of file DataSourceIngestJob.java.
Referenced by org.sleuthkit.autopsy.ingest.DataSourceIngestJob.startSecondStage().
|
private |
Definition at line 80 of file DataSourceIngestJob.java.
|
private |
Definition at line 108 of file DataSourceIngestJob.java.
|
private |
Definition at line 109 of file DataSourceIngestJob.java.
Referenced by org.sleuthkit.autopsy.ingest.DataSourceIngestJob.checkForStageCompleted().
|
staticprivate |
A data source ingest job uses the task scheduler singleton to create and queue the ingest tasks that make up the job.
Definition at line 154 of file DataSourceIngestJob.java.
Referenced by org.sleuthkit.autopsy.ingest.DataSourceIngestJob.checkForStageCompleted(), org.sleuthkit.autopsy.ingest.DataSourceIngestJob.startFirstStage(), and org.sleuthkit.autopsy.ingest.DataSourceIngestJob.startSecondStage().
Copyright © 2012-2020 Basis Technology. Generated on: Mon Jul 6 2020
This work is licensed under a
Creative Commons Attribution-Share Alike 3.0 United States License.