Making frogs purr: CRTO + CAISP + NVIDIA AI

A course review blog post? How original - well I hate 'em too... however, SEO loves 'em; so here we are.

I recently smashed out three courses to add more alphabet soup to my resume so I can make it through less-than-technical recruiters's candidate filtering. Here's a grabbag of thoughts on the courses:

CRTO: Certified Red Team Operator

The high praise that this course gets is well deserved. It's a classic assume-breach AD escalation and evasion lab, although the content covers more than that. Armed with Cobalt Strike: tweak payloads to bypass fully patched Defender and some endpoint hardening to dominate a multi-forest get-up.

The content was clean, the labs experience was smooth - and this was all with me on the old course (I had an exam voucher to use up) - I hear it's even better with the refreshed course.

I've got many a thoughts on C2s as a whole, and in my work I tend to side with the treat actors and live off the land so that I'm evading detections by using valid credentials rather than spending weeks debugging and tweaking my C2 loader because CrowdStrike or MDE pushed an update... but if you're red teaming less mature orgs and don't have access to NightHawk I can appreciate the depth the course goes to in showing you the ins and outs of Cobalt Strike.

I'm aware OSCP now covers AD content, but CRTO would show me a candidate could survive in genuine AD environments. I only learnt a thing or two from the course - it's all stuff I've hit in my time performing adversary simulation at Australia's largest companies. That's not a bite at the course - I'm not the target audience for it; I'm just here because a $previousEmployer paid for it and I thought I'd get some capital letters on my resume over a handful of weekends.

The exam was incredibly well designed. Other courses like OSCP throw you curveballs and have 400 ports open on each box that can lead you down rabbit holes. I appreciated that the exam was shaped such that at all times you knew what the next step was so that it was a test of skill and not "hurr durr try harder". I hit a few snags but admittedly that was because I wanted to do things "my" way and not the "course way" - so it was just self-imposed friction. This is the part where I post the picture of the timeline of my flags in the exam but I lost it /shrug.

One more thing, the team are... human. None of this sticky corporate support. The twice I had to contact them - they were prompt and chill as. They even refunded me on a mistake I made that they weren't required to return - any other org would have told me to suck it up; A+.

CAISP: Certified AI Security Professional

I figured it wouldn't hurt to officiate some of the security research I was doing into AI seeing as it's the "next big thing". At the time I was looking for formal courses and this was the only one around with any meat to it.

As you can see from the syllabus above it's quite comprehensive covering not just the "red" teaming of AI, but the governance and defensive sides too. There's a great deal of content (100+ videos, 50+ labs) which comes with a decent price tag (thanks for paying again $previousEmployer).

Those numbers make the course feel big but I found them waffly and very information sparse, I would just respect it a lot more if they weren't trying to pad numbers. Which is a shame because some of the labs were fantastic - e.g. handholding you through LLM and RAG concepts and implementations from first principals.

Something about Practical DevSecOps as a whole feels "off" and the sales process felt sleazy, I wouldn't do another course with them. The exam was piss easy - clearly a move to let everyone pass so they can post on LinkedIn and reap more students. There were 5 """challenges""" in the exam. All of them were solvable from copy-pasting from the labs. Three of them were "pip install this tool, run this one command, nawwww look at you you're so clever!". I maintain that you could have not once opened the content and passed the exam just by consulting Google. After all that they wanted my passport to issue my certificate PDF? HAH. It's an unproctored copy paste exam, why gatekeep with passport validation? I've got my email confirmation of passing - that's all I need thanks...

NVIDIA: Exploring Adversarial Machine Learning

NVIDIA's developed an applied AI security course. For dirt cheap, you're dropped in a Jupyter notebook in a lab with 30 hours of GPU time to do some raw hands on applied AI security. You'll work through: evasion, extraction, assessments, inversion, poisoning, LLMs and an exam that puts it all to the test.

This course is not for the faint of heart, you're up against strange python libraries, reading whitepapers, reimplementing dense mathematical formulae, debugging CUDA errors etc. But man, finishing it is bliss, nothing like a cert well earnt.

For the biggest company in the world, you'd think the content would have gone through a spell/grammar check but alas. If a ticking lab timer terrifies you as it does me there's nothing stopping you from spinning up the lab, downloading the notebook (file > new terminal > run below > right click download) and running it locally, working on the assessments locally and uploading your answers all at the end.

!tar chvfz notebook.tar.gz *

I now feel like a magician because from a raw dataset I recovered the original training data, and by either poisoning the training data or adding adversarial noise I was able to get the AI to think frogs were cats.

That's all - never stop learning.