Magi

Magi makes comprehensive digital forensic file, and comparison analysis easier.

About

Magi is a command-line tool designed to simplify digital forensic analysis, and file comparison. It allows users to compare single files, or even entire directories. All to find differences or similarities in its comprehensive analysis to go far beyond a simple comparisons by examining file metadata, strings, and entropy. It also includes advanced features for analyzing executable files, and can integrate with online threat intelligence services like VirusTotal. For easy organization and sharing, Magi can generate HTML reports, and store all analysis results in a searchable SQLite database.

So Why Use Magi?

You should use Magi because it transforms a complex, multi-step process into a single, efficient workflow. It provides a comprehensive solution for digital forensics, and file analysis that is both powerful and easy to use.

• It Simplifies Complexity: Instead of juggling multiple tools for hashing, string extraction, and metadata analysis, Magi consolidates these functions into one command-line utility, saving you time and effort.
• It Provides Deeper Insights: The tool goes beyond simple comparisons to give you meaningful insights. Its ability to analyze entropy, identify imported/exported functions, and check against online threat databases helps you uncover hidden details and potential threats that other tools might miss.
• It Enhances Collaboration and Documentation: The built-in HTML, and timeline reporting features make it simple to document your findings and share them with others, whether for a team project, a formal investigation, or simply for your own records.
• It's Designed for Everyone: The focus on being "made easy" makes powerful digital forensics accessible to a wide audience, from security professionals to developers, and researchers.

Features

• File and Directory Comparison:
  • Single File Comparison: Compare file1 and file2 to find differences or similarities.
  • Directory Comparison: Compare a baseline file (file1) against every file within a specified --target-dir.
• Comparison Modes:
  • diff (default): Identifies and reports the differences between files.
  • simm: Identifies and reports the similarities between files.
• Advanced Forensic Analysis:
  • --ssdeep: Performs fuzzy hashing analysis to find similarities between files even if their content has been slightly modified.
  • --metadata: Compares file metadata such as timestamps, size, and permissions.
  • --strings: Extracts and compares all printable strings found within the files.
  • --imports-exports: For executable files (PE), it compares the functions that are imported from and exported to other libraries.
  • --entropy: Calculates and compares the entropy (randomness) of files, which can help detect encrypted or packed data.
  • --sections: Allows you to compare specific sections of executable files (e.g., .text, .data).
  • --context: Specifies the number of bytes to display for context around found differences.

Reporting and Data Management

• Comprehensive Reports:
  • --report: Generates a detailed and easy-to-read HTML report summarizing the analysis findings.
  • --timeline: Creates a CSV timeline report for all files within a directory, useful for chronological analysis.
• Database Integration:
  • --db: Saves all analysis results to a specified SQLite database file for permanent storage.
  • --search-db: Queries a previously saved database to find records. You can search by --search-by criteria such as hash, filename, or date.

Online and Auxiliary Features

• Online Threat Intelligence:
  • --online-query: Checks file hashes against online threat intelligence services like VirusTotal and Malware Bazaar. This requires you to provide your API keys in a magi.ini configuration file.
• Verbose Output:
  • --verbose: Enables more detailed output to the console, showing the step-by-step process of the analysis.

Screenshots

This is the command line(CLI) interface.

This is a simpler demonstration of Magi.

Installation Notice

Magi uses Python3 natively, so you will need to have it installed before proceeding. Optionally, you can use the procompiled binaries.

Once you have done that follow the steps below:

Required Libraries Install Guide

You only need the following libraries manually installed to use Magi:

SSDEEP (ssdeep)

Install the ssdeep library, which is required for fuzzy hashing analysis, using pip:

pip install ssdeep

Requests (requests)

Install the requests library, which is used for making API calls to online threat intelligence services like VirusTotal and Malware Bazaar:

pip install requests

---

Note: For the --online-query feature, you must also create a magi.ini file in the same directory as the script and add your API keys.

With these libraries installed, you can proceed with the configurations.

Configuring Magi

To configure Magi, you'll need to set up the magi.ini file for its optional online features. The tool's core functionality, however, is configured directly through command-line arguments.

---

magi.ini File

The magi.ini file is essential if you want to use the --online-query feature to check files against online threat intelligence services. You must create this file in the same directory as the magi.py script and add your API keys.

The file should have a section named [API_Keys] with your personal keys for VirusTotal and Malware Bazaar.

Here’s what the file should look like:

[API_Keys]
virustotal = your_virustotal_api_key_here
malwarebazaar = your_malwarebazaar_api_key_here

You can obtain these keys by registering for a free account on the respective websites.

Command-Line Arguments

Beyond the magi.ini file, you configure the tool's behavior directly via its command-line arguments. These arguments let you specify which files to analyze, the type of comparison to perform, and what kind of reports to generate.

To see all available options, simply run the script with the --help flag:

python magi.py --help

Contributors

I welcome you to contribute code to Magi, and thank you for your contributions, feedback, and support.