Tool Release: rustymisp2sentinel

Introduction

If you want to skip reading about why I did this, click here to jump to information about the tool.

The idea behind this is quite simple. I have a former colleague that’s convinced Rust is the best thing to happen to the world since steam launched. And he’s made fun of me for my unwavering devotion to Powershell in all things.

Now, he’s a former dev and I’m nothing close to that, so maybe there’s a point to be made for Rust? My only apps have been written in Powerhell or Python - I write code mostly to automate deployments (IaC), do desired state (DSC), check posture and other security and automation-related stuff.

So I had this idea, what if I take something that I know how works (in this case the misp2sentinel solution) in a language that I know-ish (python) and have an AI agent help me vibe-code a version of that in Rust.

The thought behind it is, since I know how the original solution behaves and what it requires in terms of input and what it produces in output, I should be able to guide my AI agent (Claude Opus 4.5 in this case) to create a working solution in a language I don’t know.

So to put my theory to the test, welcome to this blog post!

The Expert Myth

I have some things to say about why I chose to do this and how I wanted to learn from it, but if you want you can just skip that and go straight to the information about the tool.

Now, this all starts by me watching the Expert Myth video by Veritasium on YouTube. It’s a recommended view, great video to watch in it’s entirity.

To summarize the video (probably without doing it justice), it shows that “experts” in the media when asked about predictions for events are often wrong. The why is simple, the events they are asked to predict happen so rarely it’s hard to build relevant expertise.

Expertise, the video concludes, is recognition. That recognition comes from the high amount of highly structured information stored in long-term memory. To build that long-term memory you require four things:

1. Valid environment
2. Many repetitions
3. Timely feedback
4. Hours of practice

I’ll expand a bit on each point.

Valid environment

Now to explain a bit deeper, the valid environment is described in the video through chess. When asked to guess positions on a board in a test where the participants can peek at the board for five seconds at a time, more experienced players require fewer peeks and can memorize more of the position each peek.

But, when you start introducing setups on the board that are impossible to reach through actual play, all players are more or less equal regardless of their experience. This tells us that the chess expert has very specific domain knowledge in chess and isn’t automatically better at memorizing things.

Many repititions

Ask experts to predict elections and they will most likely get it wrong. Sure, some might be right all the time, but that’s mostly luck and most won’t come close over a long period of time. It’s simply not enough repetitions to build expertise.

Timely feedback

When I try to run code it fails. I need to fix it. Or when I deploy an app, the login breaks. I get quick feedback. Same with video games, where I often die because of my many skills issues. This instant feedback is good, but isn’t enough on it’s own. We need to review our mistakes, see why these things happened.

In coding, this is quite simply going into the code and troubleshooting. Why is this happening? Can I remedy it, without breaking anything else? Do this enough, and you’ll start to implement better ways of writing code to avoid errors in the future.

For sports and gaming it’s a bit harder. Dying (feeding as we call it) might not teach you anything when you’re in the game. Sure, the feedback is instant but you might be missing variables. Doing reviews, looking at video of what happened and exploring what to do instead might be a good idea. The video goes into detail about why this is why athletes employ coaches, to get good timely feedback.

Hours of practice

Rise and grind baby. No, but putting hours into something you want to get better at is important. It’s not automatic. I might once have called myself a “young talent” in Rocket League, but I’m still not good at the game because I haven’t put in nearly enough time to be good.

So what does this have to do with anything?

In todays day and age there’s a lot of people building solutions (vibe-coding) claiming to create tools ready for mass consumption overnight. Sure, it’s a bit hyperbole, but one of the things this shows me is that the solution is doomed to have errors, bugs and gaping security holes.

Sure, we ask our AI coding agent to check the AI generated code for mistakes, but the truth is that as long as AI is trained on code that contains bugs, errors and zero-days it will create code that contains mistakes.

When I vibe code stuff it’s usually built to accomplish something I want or need done. It’s not something ready for mass consumption, I certainly wouldn’t charge for it. I’m painfully aware I don’t understand enough of the code to maintain and troubleshoot it without help.

So I had this idea, what if I take something that I know (ish) and have AI agent (specifically using the Claude Opus 4.5 model) help me vibe-code a version of that in Rust. And that’s this tool, rustymisp2sentinel.

flowchart TD
    C{Something I know}
    A[Python] 
    B[misp2sentinel]
    C --> A
    C --> B
    O[Expected output]
    A --> O
    B --> O 

Since I know python (well enough) and the workings of misp2sentinel in combination with the expected output from the tool, my theory is that I should be able to guide my AI agent to help me create a Rust-version that replicates the functionality of misp2sentinel with the same output.

In order to actually learn something I have to step in and do some manual work at some point, though. In my mind, having Claude vibe until a certain point was the idea. My thought was that I wanted a working binary for Linux and Windows built before I stepped in and added more features myself.

This way I had a framework to build on and I could adopt features from the MISP OpenAPI spec to my project building on that framework. Building in small increments meant I would get many repetitions and timely feedback from compilation errors (there was a metric ton of these). I started this project back in late November 2025 and have been working on it in some capacity every week since.

“How did it take you that long, are you dumb?” Yeah, yup. That’s me.

One note here is that I also tried to ship this with Azure Function support, but oh boy is running Rust on Azure Functions a can of worms. I’ll get back to that in another post.

Anyway, this bring us to the actual process of building rustymisp2sentinel.

The idea behind rustymisp2sentinel

The general outline I wanted to accomplish was something like this:

1. Take something I know - in this case the misp2sentinel python implementation
2. Have AI help me build a Rust-version of the something I know
3. Spend weeks testing, benchmarking and troubleshooting to get the solution up to a level where it performs similar to misp2sentinel
4. Have AI create a guide to teach me Rust, by using practical examples comparing parts of the code and it’s function in Rust to the counterpart in Python

So that’s what I did. I loaded up a VSCode workspace with misp2sentinel and a empty rustymisp2sentinel folder and got to work. That was a few weeks back, and I didn’t work on it every day, but it has taken some time. I’ll spare you my prompts and my manual work, but some of the issues I faced were:

1. Making sure that the solution was parsing the same amount of incidents out of the same amount of events - the Rust implementation would over 1000 events find 50.000 less indicators (out of a total 150.000 possible) than the Python implementation in the beginning
2. Trying to build on a Kali WSL was a bad idea - it started working the moment I just installed Ubuntu 24.04 WSL

Probably more but it’s been so long, I can’t hardly remember it all.

Benchmarking the two solutions

One of the issues I’ve always had an issue with when it comes to misp2sentinel is that it’s rather slow at parsing data and uploading it. Why is Rust faster? Well, it’s mostly implementation:

1. Concurrent Uploads: Rust uploads 4 batches in parallel vs Python’s sequential approach
2. Native STIX Conversion: 449,074 indicators/sec without external library overhead
3. Efficient HTTP Client: Connection pooling and async I/O with reqwest/tokio
4. Zero-Copy Parsing: Serde deserializes directly without intermediate objects

One thing I tested was running for an extended period (1000 events). I observed that the misp2sentinel solution using Python would be up to 7x slower than the Rust implementation over large datasets. When comparing 10 and 100 events, this was about 5x. Important to mention that this does not cover building, just running the .exe (I was testing on Windows). The STIX conversion was ~400x times faster in Rust than in Python.

Another thing I noticed was that the misp2sentinel solution wouldn’t refresh credentials during a run, and it was thus prone to timing out when working with large datasets. The Rust implementation didn’t have this issue as it ran for only 10 minutes (vs 70 minutes+ for Python), but it also refreshes credentials if they expire.

Testing 10 events on physical hardware locally

• MISP Instance: Self-hosted on physical hardware (no network latency) with self-signed certificate
• Filters: publish_timestamp=14d, published=true, to_ids=true
• Event Count: 10 events

Results Summary

Metric	Python	Rust	Winner
Events fetched	10	10	N/A
Indicators created	1,507	1,496	~Equal
Upload success	1,507 (100%)	1,496 (100%)	N/A
Upload errors	0	0	N/A
Total time	122.61s	23.85s	Rust 5.1x faster

Important: these are all observations from my tests running on local hardware. I have not done any tests or benchmarks on VMs or serverless where the resources available might be less, but every test was done on equal hardware.

Deduplication, STIX and other musings

So one of the issues I faced was that I was getting less indicators when running with Rust vs Python just looking at the raw numbers. This most likely has to do with how the different projects handles deduplication:

Aspect	Python	Rust
Deduplication key	Indicator ID (UUID)	STIX pattern string
Duplicate handling	Creates separate indicators	Single indicator per pattern
Unique patterns	1,496	1,496

The 11 “extra” Python indicators from the results above are duplicate STIX patterns - the same IOC appearing in multiple events (e.g., x.example.com in both “Event 1” and “Event 2” events). Rust deduplicates these since uploading the same pattern twice provides no additional threat intelligence value. I personally think this makes sense, as both events have the indicator in the TIP, while Sentinel only really need one instance of the indicator. I’m open for feedback on this.

Note: Rust supports more indicator types than Python (e.g., regkey, x509-fingerprint-*, sha3-*) but the test dataset didn’t exercise these differences significantly.

So where does learning come into the picture?

So my idea for getting some learning out of this was to have Claude generate two markdown documents:

1. RUST_INTRO.md
2. PYTHON_COMPARISON.md

The idea behind the RUST_INTRO.md was to explain Rust-concepts to the reader by comparing them to the misp2sentinel-implementation in Python. This way anyone familiar with the project could learn about Rust. This was for myself specifically.

I also wanted to give a bit more information to anyone who’s a bit more into Python than myself generally, which is why PYTHON_COMPARISON.md was created. This is ”… intended for developers familiar with the Python version who want to understand the Rust codebase.”

One thing I wanted to get out of it also was an understanding of the difference in features. You can check below for the actual features, most of which are fully implemented (feature parity). Some places the implementation is a bit different somethings exists only in Rust.

Important: A big disclaimer here is that again, this tool was created with AI under my guidance. I’ve tested this for quite some time now, but I am a bit of an idiot so please do your own tests also!

If you want to actually read these to guides yourself, you can find them in the docs-folder of the rustymisp2sentinel-repo.

Feature Parity in both solutions

Feature	Python	Rust	Notes
MISP event fetching	✅	✅	Both use REST API
STIX 2.1 conversion	✅	✅	Rust native, Python uses library
STIX Object API	✅	✅	The “new” Upload Indicators API
TLS certificate bypass	✅	✅	For self-signed certs
Event filtering (published, timestamp, tags)	✅	✅	Equivalent filter options
Batch uploads (100/batch)	✅	✅	Sentinel API limit
Rate limiting (100 req/min)	✅	✅	Sentinel API limit
Dry run mode	✅	✅	Test without uploading
TLP level mapping	✅	✅	white/green/amber/red
Confidence scoring	✅	✅	Maps MISP tags to 0-100

Different Implementation in the solutions

Feature	Python	Rust	Difference
Deduplication	By indicator ID	By pattern	Rust prevents duplicate patterns
Concurrent uploads	Sequential	4 parallel	Major performance difference
State persistence	JSON file	None (stateless)	Rust is idempotent per run
Progress reporting	Minimal	Rich progress bar	Rust shows detailed progress

Rust-Only Features

Feature	Notes
Concurrent uploads	4 parallel workers for 5x speedup
Extended indicator types	regkey, x509-fingerprint-*, sha3-*, etc.
Progress bar with ETA	Rich terminal progress display

rustymisp2sentinel

Github-repo: https://github.com/lnfernux/rustymisp2sentinel

What is it?

High-performance Rust implementation for syncing threat intelligence from MISP to Microsoft Sentinel.

How was it created?

Vibe-coded by Claude Opus 4.5 under guidance by me, the local village idiot. Use with caution.

Why was it created?

To create a solution that can handle more indicators, faster and more reliable than the Python implementation - and to try to learn a bit of Rust because some Goblin keeps telling me “it’s good it’s good”.

Goblins together strong.

So how do I get started?

The README.MD in the official repo has a pretty good guide to get started. If you don’t want to build yourself you can download one of the releases from the repo and go from there.

The SETUP_LOCAL.md has more details on the local deployment options.

Azure Function version is coming soon-ish (no promises).

How does it work in detail?

This is a general outline of the solution, but you can also dig into the ARCHITECTURE.md file to learn more about how it all works together.

flowchart TD
    MISP[("MISP Instance")]
    MISP --> API["MISP REST API<br/><i>Fetch events via /events/restSearch</i>"]
    API --> Filter["Event Filter<br/><i>Apply tag/org/timestamp filters</i>"]
    Filter --> STIX["STIX Conversion<br/><i>MISP attributes → STIX 2.1 indicators</i>"]
    STIX --> Dedup["Deduplication<br/><i>Pattern-based dedup across events</i>"]
    Dedup --> Upload["Sentinel Upload<br/><i>Upload Indicators API (batch 100/req)</i>"]
    Upload --> Sentinel[("Microsoft Sentinel")]

How does filters work in this version?

Filters are well documented within each SETUP_*.MD-file, but there’s also a document that goes into details on filters called FILTERS.MD that has the entire reference.

**You can use my MISP filter builder over at https://lnfernux.github.io/misp-filter-builder/. I wrote an announcement on this tool earlier in January.

I’ve also previously written about figuring out MISP2Sentinel filters which is still relevant.

If any filters are missing, please create an issue in the Github repository.

Does this support Graph API?

No. Depending on how the merge from Sentinel to Defender goes and how those APIs move from Azure to Microsoft 365/Graph that might change.

Does it work with the Upload Indicators API?

Yes. Which one though?

This is confusing, but Microsoft has two of the Upload Indicators APIs:

1. The original “Upload Indicators API”, which is now legacy already.
2. The “new” also called “Upload Indicators API” api, which can also be refered to “Stix Objects API”.

We support the new Stix Objects API, which is the https://api.ti.sentinel.azure.com/workspaces/{workspaceId}/threat-intelligence-stix-objects:upload?api-version={apiVersion} endpoint.

Do I need to build it myself?

No, you can download the current version from releases. When there’s changed to the main branch they should build automatically, which might break stuff. I’m an idiot so that might (is very likely to) happen and you should not deploy automatically from the new versions.