Compare commits
2 Commits
bebaeff322
...
68d749ee63
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
68d749ee63 | ||
|
|
64db5a19ed |
148
README.md
148
README.md
@ -1 +1,147 @@
|
|||||||
[Old Demo Gifs]
|
# XPlor
|
||||||
|
|
||||||
|
A Qt-based binary file format explorer and parser using a custom domain-specific language (XScript) for defining file structures.
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
XPlor is a desktop application designed to explore, parse, and visualize binary file formats from video games. It uses XScript, a custom DSL (Domain-Specific Language), to define how binary files should be interpreted, making it easy to add support for new file formats without modifying the core application code.
|
||||||
|
|
||||||
|
## Features
|
||||||
|
|
||||||
|
- **XScript DSL**: Define binary file structures using a readable, declarative scripting language
|
||||||
|
- **Dynamic File Type Detection**: Automatically identifies file types based on extension and magic bytes
|
||||||
|
- **Tree-Based UI**: Hierarchical view of parsed file structures
|
||||||
|
- **Image Preview**: Built-in preview for texture assets (TGA, DXT, Xbox 360 formats)
|
||||||
|
- **Extensible**: Add support for new file formats by creating XScript definition files
|
||||||
|
- **Multi-Platform Support**: Parse files from PC, Xbox 360, PS3, and other platforms
|
||||||
|
|
||||||
|
## Supported Games/Formats
|
||||||
|
|
||||||
|
### Call of Duty Series
|
||||||
|
- Fast Files (`.ff`)
|
||||||
|
- Various asset types: materials, textures, models, sounds, menus, weapons, etc.
|
||||||
|
|
||||||
|
### Rebellion Asura Engine (Sniper Elite V2)
|
||||||
|
- Archive files (`.asrbe`, `.tsBE`, `.mapBE`, etc.)
|
||||||
|
- Texture archives
|
||||||
|
- Various chunk types
|
||||||
|
|
||||||
|
## Project Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
XPlor/
|
||||||
|
├── app/ # Main application
|
||||||
|
│ ├── mainwindow.* # Main window UI and logic
|
||||||
|
│ ├── imagepreviewwidget.* # Texture preview widget
|
||||||
|
│ └── xtreewidget.* # Tree view components
|
||||||
|
├── libs/
|
||||||
|
│ ├── core/ # Core utilities and managers
|
||||||
|
│ ├── dsl/ # XScript DSL engine
|
||||||
|
│ │ ├── lexer.* # Tokenizer
|
||||||
|
│ │ ├── parser.* # AST parser
|
||||||
|
│ │ ├── interpreter.* # Runtime interpreter
|
||||||
|
│ │ └── typeregistry.* # Type management
|
||||||
|
│ ├── compression/ # LZO compression support
|
||||||
|
│ └── encryption/ # Salsa20, SHA1 encryption
|
||||||
|
├── definitions/ # XScript format definitions
|
||||||
|
│ ├── cod/ # Call of Duty formats
|
||||||
|
│ └── asura/ # Asura Engine formats
|
||||||
|
├── tools/ # Command-line utilities
|
||||||
|
└── third_party/ # External dependencies
|
||||||
|
```
|
||||||
|
|
||||||
|
## Building
|
||||||
|
|
||||||
|
### Prerequisites
|
||||||
|
|
||||||
|
- Qt 6.x (tested with 6.10.0)
|
||||||
|
- MSVC 2022 (Windows)
|
||||||
|
- CMake or qmake
|
||||||
|
|
||||||
|
### Build Steps
|
||||||
|
|
||||||
|
1. Open `XPlor.pro` in Qt Creator
|
||||||
|
2. Configure the project for your Qt kit
|
||||||
|
3. Build the project
|
||||||
|
|
||||||
|
Or from command line:
|
||||||
|
```bash
|
||||||
|
qmake XPlor.pro
|
||||||
|
make # or nmake/jom on Windows
|
||||||
|
```
|
||||||
|
|
||||||
|
## XScript Language
|
||||||
|
|
||||||
|
XScript is a domain-specific language for defining binary file structures. Here's an example:
|
||||||
|
|
||||||
|
```xscript
|
||||||
|
type fastfile [root, display="Fast File"] byteorder LE
|
||||||
|
{
|
||||||
|
criteria {
|
||||||
|
require _ext == "ff";
|
||||||
|
}
|
||||||
|
|
||||||
|
// Read header
|
||||||
|
magic = ascii(read(8));
|
||||||
|
version = u32;
|
||||||
|
|
||||||
|
if (version == 5) {
|
||||||
|
platform = "PC";
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse nested structures
|
||||||
|
asset_count = u32;
|
||||||
|
repeat asset_count {
|
||||||
|
asset = parse_here("asset");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Key Features
|
||||||
|
|
||||||
|
- **Type definitions** with byte order specification
|
||||||
|
- **Criteria blocks** for file type detection
|
||||||
|
- **Built-in functions**: `read()`, `u32`, `ascii()`, `cstring()`, etc.
|
||||||
|
- **Control flow**: `if/else`, `while`, `repeat`, `for` loops
|
||||||
|
- **Pipeline syntax**: `data | decompress("zlib") | parse("asset")`
|
||||||
|
- **UI annotations**: `[ui, display="Name", readonly]`
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
1. Launch XPlor
|
||||||
|
2. Drag and drop a supported file onto the window, or use File > Open
|
||||||
|
3. The file will be parsed and displayed in a tree structure
|
||||||
|
4. Click on tree items to view their properties
|
||||||
|
5. For texture assets, a preview will be shown
|
||||||
|
|
||||||
|
## Adding New Format Support
|
||||||
|
|
||||||
|
1. Create a new `.xscript` file in `definitions/`
|
||||||
|
2. Define your root type with `[root]` attribute
|
||||||
|
3. Add criteria for file detection
|
||||||
|
4. Define the binary structure using XScript syntax
|
||||||
|
5. Restart XPlor to load the new definition
|
||||||
|
|
||||||
|
## Libraries
|
||||||
|
|
||||||
|
- **Qt 6**: UI framework
|
||||||
|
- **miniLZO**: LZO compression
|
||||||
|
- **Salsa20**: Stream cipher encryption
|
||||||
|
- **zlib**: Deflate compression
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
This project is for educational and research purposes.
|
||||||
|
|
||||||
|
## Contributing
|
||||||
|
|
||||||
|
Contributions are welcome! Please feel free to submit pull requests for:
|
||||||
|
- New file format definitions
|
||||||
|
- Bug fixes
|
||||||
|
- Feature improvements
|
||||||
|
- Documentation
|
||||||
|
|
||||||
|
## Acknowledgments
|
||||||
|
|
||||||
|
- Xbox 360 SDK documentation for texture untiling algorithms
|
||||||
|
- Various game modding communities for format documentation
|
||||||
|
|||||||
@ -297,8 +297,100 @@ MainWindow::MainWindow(QWidget *parent)
|
|||||||
// TODO: Implement New action
|
// TODO: Implement New action
|
||||||
});
|
});
|
||||||
|
|
||||||
connect(actionOpen, &QAction::triggered, this, []() {
|
connect(actionOpen, &QAction::triggered, this, [this]() {
|
||||||
// TODO: Implement Open action
|
// Build filter string from loaded definitions
|
||||||
|
QMap<QString, QString> exts = mTypeRegistry.supportedExtensions();
|
||||||
|
|
||||||
|
QStringList filterParts;
|
||||||
|
QStringList allExts;
|
||||||
|
|
||||||
|
for (auto it = exts.begin(); it != exts.end(); ++it) {
|
||||||
|
const QString& ext = it.key();
|
||||||
|
const QString& displayName = it.value();
|
||||||
|
filterParts.append(QString("%1 (*.%2)").arg(displayName, ext));
|
||||||
|
allExts.append(QString("*.%1").arg(ext));
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sort filters alphabetically by display name
|
||||||
|
filterParts.sort(Qt::CaseInsensitive);
|
||||||
|
|
||||||
|
// Add "All Supported Files" at the beginning
|
||||||
|
QString allFilter = QString("All Supported Files (%1)").arg(allExts.join(" "));
|
||||||
|
filterParts.prepend(allFilter);
|
||||||
|
|
||||||
|
// Add "All Files" at the end
|
||||||
|
filterParts.append("All Files (*)");
|
||||||
|
|
||||||
|
QString filter = filterParts.join(";;");
|
||||||
|
|
||||||
|
QStringList filePaths = QFileDialog::getOpenFileNames(
|
||||||
|
this,
|
||||||
|
"Open File",
|
||||||
|
QString(),
|
||||||
|
filter
|
||||||
|
);
|
||||||
|
|
||||||
|
for (const QString& path : filePaths) {
|
||||||
|
const QString fileName = QFileInfo(path).fileName();
|
||||||
|
|
||||||
|
QFile inputFile(path);
|
||||||
|
if (!inputFile.open(QIODevice::ReadOnly)) {
|
||||||
|
LogManager::instance().addError(QString("Failed to open: %1").arg(path));
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
LogManager::instance().addEntry(QString("[FILE] Analyzing: %1").arg(fileName));
|
||||||
|
|
||||||
|
const QString rootType = mTypeRegistry.chooseType(&inputFile, path);
|
||||||
|
LogManager::instance().addEntry(QString("[FILE] Matched type '%1' for file: %2").arg(rootType).arg(fileName));
|
||||||
|
|
||||||
|
if (rootType.isEmpty()) {
|
||||||
|
LogManager::instance().addError(QString("No matching definition for: %1").arg(path));
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
inputFile.seek(0);
|
||||||
|
LogManager::instance().addLine();
|
||||||
|
LogManager::instance().addEntry(QString("========== PARSING %1 ==========").arg(fileName));
|
||||||
|
LogManager::instance().addLine();
|
||||||
|
|
||||||
|
const qint64 fileSize = inputFile.size();
|
||||||
|
QProgressDialog progress(QString("Parsing %1...").arg(fileName), "Cancel", 0, static_cast<int>(fileSize), this);
|
||||||
|
progress.setWindowModality(Qt::WindowModal);
|
||||||
|
progress.setMinimumDuration(500);
|
||||||
|
|
||||||
|
QVariantMap rootVars;
|
||||||
|
try {
|
||||||
|
rootVars = mTypeRegistry.parse(rootType, &inputFile, path, [this, &progress](qint64 pos, qint64 size) {
|
||||||
|
progress.setValue(static_cast<int>(pos));
|
||||||
|
QApplication::processEvents();
|
||||||
|
});
|
||||||
|
} catch (const std::exception& e) {
|
||||||
|
LogManager::instance().addLine();
|
||||||
|
LogManager::instance().addError(QString("========== PARSE ERROR in %1 ==========").arg(fileName));
|
||||||
|
LogManager::instance().addError(QString("Error: %1").arg(e.what()));
|
||||||
|
LogManager::instance().addError(QString("File position: 0x%1").arg(inputFile.pos(), 0, 16));
|
||||||
|
LogManager::instance().addLine();
|
||||||
|
QMessageBox::critical(this, "Parse Error",
|
||||||
|
QString("Failed to parse %1:\n\n%2").arg(fileName).arg(e.what()));
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
LogManager::instance().addLine();
|
||||||
|
LogManager::instance().addEntry(QString("========== FINISHED %1 ==========").arg(fileName));
|
||||||
|
LogManager::instance().addLine();
|
||||||
|
|
||||||
|
if (!mOpenedFilePaths.contains(path)) {
|
||||||
|
mOpenedFilePaths.append(path);
|
||||||
|
}
|
||||||
|
|
||||||
|
XTreeWidgetItem* cat = ensureTypeCategoryRoot(rootType);
|
||||||
|
auto* rootInst = addInstanceNode(cat, fileName, rootType, rootVars);
|
||||||
|
routeNestedObjects(rootInst, rootVars);
|
||||||
|
|
||||||
|
cat->setExpanded(true);
|
||||||
|
mTreeWidget->setCurrentItem(rootInst);
|
||||||
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
connect(actionOpenFolder, &QAction::triggered, this, []() {
|
connect(actionOpenFolder, &QAction::triggered, this, []() {
|
||||||
|
|||||||
@ -39,6 +39,46 @@ QStringList TypeRegistry::typeNames() const {
|
|||||||
return m_module.types.keys();
|
return m_module.types.keys();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
QMap<QString, QString> TypeRegistry::supportedExtensions() const {
|
||||||
|
QMap<QString, QString> result;
|
||||||
|
|
||||||
|
for (auto it = m_module.types.begin(); it != m_module.types.end(); ++it) {
|
||||||
|
const TypeDef& td = it.value();
|
||||||
|
if (!td.isRoot) continue;
|
||||||
|
|
||||||
|
// Scan criteria for `require _ext == "xxx"` patterns
|
||||||
|
for (const auto& stmtPtr : td.criteria) {
|
||||||
|
if (!std::holds_alternative<Stmt::Require>(stmtPtr->node)) continue;
|
||||||
|
|
||||||
|
const auto& req = std::get<Stmt::Require>(stmtPtr->node);
|
||||||
|
if (!std::holds_alternative<Expr::Binary>(req.cond->node)) continue;
|
||||||
|
|
||||||
|
const auto& bin = std::get<Expr::Binary>(req.cond->node);
|
||||||
|
if (bin.op != "==") continue;
|
||||||
|
|
||||||
|
// Check for _ext == "xxx" or "xxx" == _ext
|
||||||
|
QString ext;
|
||||||
|
bool lhsIsExt = std::holds_alternative<Expr::Var>(bin.lhs->node) &&
|
||||||
|
std::get<Expr::Var>(bin.lhs->node).name == "_ext";
|
||||||
|
bool rhsIsExt = std::holds_alternative<Expr::Var>(bin.rhs->node) &&
|
||||||
|
std::get<Expr::Var>(bin.rhs->node).name == "_ext";
|
||||||
|
|
||||||
|
if (lhsIsExt && std::holds_alternative<Expr::String>(bin.rhs->node)) {
|
||||||
|
ext = std::get<Expr::String>(bin.rhs->node).v;
|
||||||
|
} else if (rhsIsExt && std::holds_alternative<Expr::String>(bin.lhs->node)) {
|
||||||
|
ext = std::get<Expr::String>(bin.lhs->node).v;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!ext.isEmpty()) {
|
||||||
|
QString displayName = td.display.isEmpty() ? td.name : td.display;
|
||||||
|
result.insert(ext.toLower(), displayName);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
void TypeRegistry::collectTypeRefsFromExpr(const Expr& expr, QSet<QString>& refs) const {
|
void TypeRegistry::collectTypeRefsFromExpr(const Expr& expr, QSet<QString>& refs) const {
|
||||||
if (std::holds_alternative<Expr::Call>(expr.node)) {
|
if (std::holds_alternative<Expr::Call>(expr.node)) {
|
||||||
const auto& call = std::get<Expr::Call>(expr.node);
|
const auto& call = std::get<Expr::Call>(expr.node);
|
||||||
|
|||||||
@ -19,6 +19,9 @@ public:
|
|||||||
bool hasType(const QString& typeName) const;
|
bool hasType(const QString& typeName) const;
|
||||||
QStringList typeNames() const;
|
QStringList typeNames() const;
|
||||||
|
|
||||||
|
// Returns map of extension -> display name for all root types
|
||||||
|
QMap<QString, QString> supportedExtensions() const;
|
||||||
|
|
||||||
// Validate all type references - returns list of errors, empty if valid
|
// Validate all type references - returns list of errors, empty if valid
|
||||||
QStringList validateTypeReferences() const;
|
QStringList validateTypeReferences() const;
|
||||||
|
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user