Compare commits
No commits in common. "68d749ee6390c3ef594d8bc23bd7e0af88de2471" and "bebaeff322ac3ed3c3157d276bcdd62b3451d4cc" have entirely different histories.
68d749ee63
...
bebaeff322
148
README.md
148
README.md
@ -1,147 +1 @@
|
|||||||
# XPlor
|
[Old Demo Gifs]
|
||||||
|
|
||||||
A Qt-based binary file format explorer and parser using a custom domain-specific language (XScript) for defining file structures.
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
XPlor is a desktop application designed to explore, parse, and visualize binary file formats from video games. It uses XScript, a custom DSL (Domain-Specific Language), to define how binary files should be interpreted, making it easy to add support for new file formats without modifying the core application code.
|
|
||||||
|
|
||||||
## Features
|
|
||||||
|
|
||||||
- **XScript DSL**: Define binary file structures using a readable, declarative scripting language
|
|
||||||
- **Dynamic File Type Detection**: Automatically identifies file types based on extension and magic bytes
|
|
||||||
- **Tree-Based UI**: Hierarchical view of parsed file structures
|
|
||||||
- **Image Preview**: Built-in preview for texture assets (TGA, DXT, Xbox 360 formats)
|
|
||||||
- **Extensible**: Add support for new file formats by creating XScript definition files
|
|
||||||
- **Multi-Platform Support**: Parse files from PC, Xbox 360, PS3, and other platforms
|
|
||||||
|
|
||||||
## Supported Games/Formats
|
|
||||||
|
|
||||||
### Call of Duty Series
|
|
||||||
- Fast Files (`.ff`)
|
|
||||||
- Various asset types: materials, textures, models, sounds, menus, weapons, etc.
|
|
||||||
|
|
||||||
### Rebellion Asura Engine (Sniper Elite V2)
|
|
||||||
- Archive files (`.asrbe`, `.tsBE`, `.mapBE`, etc.)
|
|
||||||
- Texture archives
|
|
||||||
- Various chunk types
|
|
||||||
|
|
||||||
## Project Structure
|
|
||||||
|
|
||||||
```
|
|
||||||
XPlor/
|
|
||||||
├── app/ # Main application
|
|
||||||
│ ├── mainwindow.* # Main window UI and logic
|
|
||||||
│ ├── imagepreviewwidget.* # Texture preview widget
|
|
||||||
│ └── xtreewidget.* # Tree view components
|
|
||||||
├── libs/
|
|
||||||
│ ├── core/ # Core utilities and managers
|
|
||||||
│ ├── dsl/ # XScript DSL engine
|
|
||||||
│ │ ├── lexer.* # Tokenizer
|
|
||||||
│ │ ├── parser.* # AST parser
|
|
||||||
│ │ ├── interpreter.* # Runtime interpreter
|
|
||||||
│ │ └── typeregistry.* # Type management
|
|
||||||
│ ├── compression/ # LZO compression support
|
|
||||||
│ └── encryption/ # Salsa20, SHA1 encryption
|
|
||||||
├── definitions/ # XScript format definitions
|
|
||||||
│ ├── cod/ # Call of Duty formats
|
|
||||||
│ └── asura/ # Asura Engine formats
|
|
||||||
├── tools/ # Command-line utilities
|
|
||||||
└── third_party/ # External dependencies
|
|
||||||
```
|
|
||||||
|
|
||||||
## Building
|
|
||||||
|
|
||||||
### Prerequisites
|
|
||||||
|
|
||||||
- Qt 6.x (tested with 6.10.0)
|
|
||||||
- MSVC 2022 (Windows)
|
|
||||||
- CMake or qmake
|
|
||||||
|
|
||||||
### Build Steps
|
|
||||||
|
|
||||||
1. Open `XPlor.pro` in Qt Creator
|
|
||||||
2. Configure the project for your Qt kit
|
|
||||||
3. Build the project
|
|
||||||
|
|
||||||
Or from command line:
|
|
||||||
```bash
|
|
||||||
qmake XPlor.pro
|
|
||||||
make # or nmake/jom on Windows
|
|
||||||
```
|
|
||||||
|
|
||||||
## XScript Language
|
|
||||||
|
|
||||||
XScript is a domain-specific language for defining binary file structures. Here's an example:
|
|
||||||
|
|
||||||
```xscript
|
|
||||||
type fastfile [root, display="Fast File"] byteorder LE
|
|
||||||
{
|
|
||||||
criteria {
|
|
||||||
require _ext == "ff";
|
|
||||||
}
|
|
||||||
|
|
||||||
// Read header
|
|
||||||
magic = ascii(read(8));
|
|
||||||
version = u32;
|
|
||||||
|
|
||||||
if (version == 5) {
|
|
||||||
platform = "PC";
|
|
||||||
}
|
|
||||||
|
|
||||||
// Parse nested structures
|
|
||||||
asset_count = u32;
|
|
||||||
repeat asset_count {
|
|
||||||
asset = parse_here("asset");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Key Features
|
|
||||||
|
|
||||||
- **Type definitions** with byte order specification
|
|
||||||
- **Criteria blocks** for file type detection
|
|
||||||
- **Built-in functions**: `read()`, `u32`, `ascii()`, `cstring()`, etc.
|
|
||||||
- **Control flow**: `if/else`, `while`, `repeat`, `for` loops
|
|
||||||
- **Pipeline syntax**: `data | decompress("zlib") | parse("asset")`
|
|
||||||
- **UI annotations**: `[ui, display="Name", readonly]`
|
|
||||||
|
|
||||||
## Usage
|
|
||||||
|
|
||||||
1. Launch XPlor
|
|
||||||
2. Drag and drop a supported file onto the window, or use File > Open
|
|
||||||
3. The file will be parsed and displayed in a tree structure
|
|
||||||
4. Click on tree items to view their properties
|
|
||||||
5. For texture assets, a preview will be shown
|
|
||||||
|
|
||||||
## Adding New Format Support
|
|
||||||
|
|
||||||
1. Create a new `.xscript` file in `definitions/`
|
|
||||||
2. Define your root type with `[root]` attribute
|
|
||||||
3. Add criteria for file detection
|
|
||||||
4. Define the binary structure using XScript syntax
|
|
||||||
5. Restart XPlor to load the new definition
|
|
||||||
|
|
||||||
## Libraries
|
|
||||||
|
|
||||||
- **Qt 6**: UI framework
|
|
||||||
- **miniLZO**: LZO compression
|
|
||||||
- **Salsa20**: Stream cipher encryption
|
|
||||||
- **zlib**: Deflate compression
|
|
||||||
|
|
||||||
## License
|
|
||||||
|
|
||||||
This project is for educational and research purposes.
|
|
||||||
|
|
||||||
## Contributing
|
|
||||||
|
|
||||||
Contributions are welcome! Please feel free to submit pull requests for:
|
|
||||||
- New file format definitions
|
|
||||||
- Bug fixes
|
|
||||||
- Feature improvements
|
|
||||||
- Documentation
|
|
||||||
|
|
||||||
## Acknowledgments
|
|
||||||
|
|
||||||
- Xbox 360 SDK documentation for texture untiling algorithms
|
|
||||||
- Various game modding communities for format documentation
|
|
||||||
|
|||||||
@ -297,100 +297,8 @@ MainWindow::MainWindow(QWidget *parent)
|
|||||||
// TODO: Implement New action
|
// TODO: Implement New action
|
||||||
});
|
});
|
||||||
|
|
||||||
connect(actionOpen, &QAction::triggered, this, [this]() {
|
connect(actionOpen, &QAction::triggered, this, []() {
|
||||||
// Build filter string from loaded definitions
|
// TODO: Implement Open action
|
||||||
QMap<QString, QString> exts = mTypeRegistry.supportedExtensions();
|
|
||||||
|
|
||||||
QStringList filterParts;
|
|
||||||
QStringList allExts;
|
|
||||||
|
|
||||||
for (auto it = exts.begin(); it != exts.end(); ++it) {
|
|
||||||
const QString& ext = it.key();
|
|
||||||
const QString& displayName = it.value();
|
|
||||||
filterParts.append(QString("%1 (*.%2)").arg(displayName, ext));
|
|
||||||
allExts.append(QString("*.%1").arg(ext));
|
|
||||||
}
|
|
||||||
|
|
||||||
// Sort filters alphabetically by display name
|
|
||||||
filterParts.sort(Qt::CaseInsensitive);
|
|
||||||
|
|
||||||
// Add "All Supported Files" at the beginning
|
|
||||||
QString allFilter = QString("All Supported Files (%1)").arg(allExts.join(" "));
|
|
||||||
filterParts.prepend(allFilter);
|
|
||||||
|
|
||||||
// Add "All Files" at the end
|
|
||||||
filterParts.append("All Files (*)");
|
|
||||||
|
|
||||||
QString filter = filterParts.join(";;");
|
|
||||||
|
|
||||||
QStringList filePaths = QFileDialog::getOpenFileNames(
|
|
||||||
this,
|
|
||||||
"Open File",
|
|
||||||
QString(),
|
|
||||||
filter
|
|
||||||
);
|
|
||||||
|
|
||||||
for (const QString& path : filePaths) {
|
|
||||||
const QString fileName = QFileInfo(path).fileName();
|
|
||||||
|
|
||||||
QFile inputFile(path);
|
|
||||||
if (!inputFile.open(QIODevice::ReadOnly)) {
|
|
||||||
LogManager::instance().addError(QString("Failed to open: %1").arg(path));
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
|
|
||||||
LogManager::instance().addEntry(QString("[FILE] Analyzing: %1").arg(fileName));
|
|
||||||
|
|
||||||
const QString rootType = mTypeRegistry.chooseType(&inputFile, path);
|
|
||||||
LogManager::instance().addEntry(QString("[FILE] Matched type '%1' for file: %2").arg(rootType).arg(fileName));
|
|
||||||
|
|
||||||
if (rootType.isEmpty()) {
|
|
||||||
LogManager::instance().addError(QString("No matching definition for: %1").arg(path));
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
|
|
||||||
inputFile.seek(0);
|
|
||||||
LogManager::instance().addLine();
|
|
||||||
LogManager::instance().addEntry(QString("========== PARSING %1 ==========").arg(fileName));
|
|
||||||
LogManager::instance().addLine();
|
|
||||||
|
|
||||||
const qint64 fileSize = inputFile.size();
|
|
||||||
QProgressDialog progress(QString("Parsing %1...").arg(fileName), "Cancel", 0, static_cast<int>(fileSize), this);
|
|
||||||
progress.setWindowModality(Qt::WindowModal);
|
|
||||||
progress.setMinimumDuration(500);
|
|
||||||
|
|
||||||
QVariantMap rootVars;
|
|
||||||
try {
|
|
||||||
rootVars = mTypeRegistry.parse(rootType, &inputFile, path, [this, &progress](qint64 pos, qint64 size) {
|
|
||||||
progress.setValue(static_cast<int>(pos));
|
|
||||||
QApplication::processEvents();
|
|
||||||
});
|
|
||||||
} catch (const std::exception& e) {
|
|
||||||
LogManager::instance().addLine();
|
|
||||||
LogManager::instance().addError(QString("========== PARSE ERROR in %1 ==========").arg(fileName));
|
|
||||||
LogManager::instance().addError(QString("Error: %1").arg(e.what()));
|
|
||||||
LogManager::instance().addError(QString("File position: 0x%1").arg(inputFile.pos(), 0, 16));
|
|
||||||
LogManager::instance().addLine();
|
|
||||||
QMessageBox::critical(this, "Parse Error",
|
|
||||||
QString("Failed to parse %1:\n\n%2").arg(fileName).arg(e.what()));
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
|
|
||||||
LogManager::instance().addLine();
|
|
||||||
LogManager::instance().addEntry(QString("========== FINISHED %1 ==========").arg(fileName));
|
|
||||||
LogManager::instance().addLine();
|
|
||||||
|
|
||||||
if (!mOpenedFilePaths.contains(path)) {
|
|
||||||
mOpenedFilePaths.append(path);
|
|
||||||
}
|
|
||||||
|
|
||||||
XTreeWidgetItem* cat = ensureTypeCategoryRoot(rootType);
|
|
||||||
auto* rootInst = addInstanceNode(cat, fileName, rootType, rootVars);
|
|
||||||
routeNestedObjects(rootInst, rootVars);
|
|
||||||
|
|
||||||
cat->setExpanded(true);
|
|
||||||
mTreeWidget->setCurrentItem(rootInst);
|
|
||||||
}
|
|
||||||
});
|
});
|
||||||
|
|
||||||
connect(actionOpenFolder, &QAction::triggered, this, []() {
|
connect(actionOpenFolder, &QAction::triggered, this, []() {
|
||||||
|
|||||||
@ -39,46 +39,6 @@ QStringList TypeRegistry::typeNames() const {
|
|||||||
return m_module.types.keys();
|
return m_module.types.keys();
|
||||||
}
|
}
|
||||||
|
|
||||||
QMap<QString, QString> TypeRegistry::supportedExtensions() const {
|
|
||||||
QMap<QString, QString> result;
|
|
||||||
|
|
||||||
for (auto it = m_module.types.begin(); it != m_module.types.end(); ++it) {
|
|
||||||
const TypeDef& td = it.value();
|
|
||||||
if (!td.isRoot) continue;
|
|
||||||
|
|
||||||
// Scan criteria for `require _ext == "xxx"` patterns
|
|
||||||
for (const auto& stmtPtr : td.criteria) {
|
|
||||||
if (!std::holds_alternative<Stmt::Require>(stmtPtr->node)) continue;
|
|
||||||
|
|
||||||
const auto& req = std::get<Stmt::Require>(stmtPtr->node);
|
|
||||||
if (!std::holds_alternative<Expr::Binary>(req.cond->node)) continue;
|
|
||||||
|
|
||||||
const auto& bin = std::get<Expr::Binary>(req.cond->node);
|
|
||||||
if (bin.op != "==") continue;
|
|
||||||
|
|
||||||
// Check for _ext == "xxx" or "xxx" == _ext
|
|
||||||
QString ext;
|
|
||||||
bool lhsIsExt = std::holds_alternative<Expr::Var>(bin.lhs->node) &&
|
|
||||||
std::get<Expr::Var>(bin.lhs->node).name == "_ext";
|
|
||||||
bool rhsIsExt = std::holds_alternative<Expr::Var>(bin.rhs->node) &&
|
|
||||||
std::get<Expr::Var>(bin.rhs->node).name == "_ext";
|
|
||||||
|
|
||||||
if (lhsIsExt && std::holds_alternative<Expr::String>(bin.rhs->node)) {
|
|
||||||
ext = std::get<Expr::String>(bin.rhs->node).v;
|
|
||||||
} else if (rhsIsExt && std::holds_alternative<Expr::String>(bin.lhs->node)) {
|
|
||||||
ext = std::get<Expr::String>(bin.lhs->node).v;
|
|
||||||
}
|
|
||||||
|
|
||||||
if (!ext.isEmpty()) {
|
|
||||||
QString displayName = td.display.isEmpty() ? td.name : td.display;
|
|
||||||
result.insert(ext.toLower(), displayName);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return result;
|
|
||||||
}
|
|
||||||
|
|
||||||
void TypeRegistry::collectTypeRefsFromExpr(const Expr& expr, QSet<QString>& refs) const {
|
void TypeRegistry::collectTypeRefsFromExpr(const Expr& expr, QSet<QString>& refs) const {
|
||||||
if (std::holds_alternative<Expr::Call>(expr.node)) {
|
if (std::holds_alternative<Expr::Call>(expr.node)) {
|
||||||
const auto& call = std::get<Expr::Call>(expr.node);
|
const auto& call = std::get<Expr::Call>(expr.node);
|
||||||
|
|||||||
@ -19,9 +19,6 @@ public:
|
|||||||
bool hasType(const QString& typeName) const;
|
bool hasType(const QString& typeName) const;
|
||||||
QStringList typeNames() const;
|
QStringList typeNames() const;
|
||||||
|
|
||||||
// Returns map of extension -> display name for all root types
|
|
||||||
QMap<QString, QString> supportedExtensions() const;
|
|
||||||
|
|
||||||
// Validate all type references - returns list of errors, empty if valid
|
// Validate all type references - returns list of errors, empty if valid
|
||||||
QStringList validateTypeReferences() const;
|
QStringList validateTypeReferences() const;
|
||||||
|
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user