Transcript
XO Ruby Atlanta 2025 - SAST and Sensibility: A Rubyist’s Guide to Static... by Kylie Stradley
From Confreaks · Episode page · Watch on YouTube
· 32 min · 5,101 words · 39 timestamped sections
This is the full, timestamped transcript of "XO Ruby Atlanta 2025 - SAST and Sensibility: A Rubyist’s Guide to Static... by Kylie Stradley", an episode of Confreaks published January 6, 2026 (32 min, about 5,101 words). It mentions 12 people, works and ideas, including CodeQL, GitHub, Breakman, OpenGP, Rails.
What is discussed, and when
Notable quotes from "XO Ruby Atlanta 2025 - SAST and Sensibility: A Rubyist’s Guide to Static... by Kylie Stradley"
Stradley. I'm a product security engineer working remotely right here in Atlanta, Georgia for GitHub. as a product security engineer, I do a couple different things. right now…
about GitHub · atThe name of this talk is Sast and Sensibility and it's loosely inspired by a novel Sense and Sensibility by Jane Austin. and it is a story about two sisters. One who represents…
about Sense and Sensibility · atCondition to every single method test in every single controller? I don't know, maybe. But a tool like CodeQL or open GP with taint tracking can trace the flow from params Q to eval and flag it as unsafe.
about CodeQL · atPositives. This is the most powerful detection method, but it's also the slowest by a lot. OpenGP, which is a fork of a private product called Sim Grep, builds an abstract…
about OpenGP · atRubocop it is abstract syntax tree aware. It's more focused towards quality but it does implement some security rules that you can add and you could also write your own security rules if.
about Rubocop · atDevkim I wanted to love you. DevSkim catches the dangerous deserialization, but none of the SQL or command injection patterns. You would have to write those Ruby rules yourself.
about DevSkim · at
Who and what is mentioned in this episode?
People
- Kylie Stradley — Person · 1 mention · first at
I'm going to attempt to jam a bunch of technical content right into your afternoon slump brains. My name is Kylie Stradley. I'm a product security engineer working remotely…
- Jane Austen — Person · 1 mention
Books & papers
- Sense and Sensibility — Book · 2 mentions · first at
Companies & orgs
- GitHub — Company · 8 mentions · first at
Products & tech
- CodeQL — Product · 9 mentions · first at
- Breakman — Product · 8 mentions · first at
Won't catch is that this is an injection sync. Static analysis tools like breakman or open grab can flag this as a potential cross-sight scripting vulnerability because…
- OpenGP — Product · 5 mentions · first at
- Rubocop — Product · 4 mentions · first at
- DevSkim — Product · 1 mention · first at
Concepts & topics
Full transcript with timestamps
Um, thanks so much, Jim. I'm really excited to be here. I never got to speak um at earlier conference but uh if you have an opportunity to what I was told is if you have an opportunity to attend an event organized by Jim like absolutely go. So thank you so much for having me here. Um good afternoon uh our small but mighty remainders. [gasps] So, I hope that you had a moment uh to hit up like Chrome Yellow Coffee or Little Tart Bake Shop, which are nearby, so you could get some coffee. If you couldn't and you're here tomorrow, I recommend that you get one on the way out. Um if you are uncaffeinated, life's tough, man. Um cuz [gasps and laughter] I'm going to attempt to jam um a bunch of technical content uh right into your afternoon slump brains. My name is Kylie Stradley. I'm a product security engineer working remotely right here in Atlanta, Georgia for GitHub. Um, as a product security engineer, I do a couple
different things. Um, right now my big focus is on detection. So, I attempt to detect and solve for classes of vulnerabilities at GitHub scale, which is huge. Um, I do that for all of the services that make up github.com, code spaces copilot actions. Um, and then also we do some scanning on some open source projects. Uh, if you're the owner of an open source project and you're part of the GitHub stars program, I'm monitoring your code and you might receive a message from me. We'll see depending on how bad the vulnerability is. Um, I wake up, you know, the five days a week that they pay me to be there and choose to be a security engineer and write security code.
You do not have that choice. Um, at some point in your career, if you haven't done so already, you will write security code. Now what you do have a choice in is when you write some of that security code and if you use static analysis security tooling you can try you can write your most of your security code during 9 to5 um and not after hours and not as a surprise. So if you like to know about your work in advance um and you are a Rails developer Ruby developer you might be interested in static analysis security tooling.
The name of this talk is Sast and Sensibility and it's loosely inspired by a novel Sense and Sensibility by Jane Austin. Um, and it is a story about two sisters. One who represents sense, logic, restraint, and practicality, and another who embodies sensibility, emotion intuition spontaneity. Uh when the novel was first published, it was widely interpreted as a critique of the previous era's like excessive obsession with sensibility and like romance, right? But um modern interpretations, spoiler alert, including by this speaker, see things differently. Um I think it's a story about balance. Uh my reading is that both sisters grow by learning from each other and they incorporate sense and sensibility into their mindsets.
In security, sense is essential. Um, we need deep analysis. We have to make thoughtful trade-offs and sometimes we have to use complex tools like static analysis, security testing. But as a Rubist, I value sensibility. Tools that are easy to use, um, often package as gems, easy to set up, uh, quick to set up, and that fit naturally into my workflow. So, I'm going to talk about some static analysis primitives, the tools that implement them, and how I think you could incorporate these tools into your development life cycle without sacrificing too much of the Ruby programming values that we all cherish, balancing SAS and sensibility and hopefully doing some scheduled instead of surprise security work.
One of the biggest advantage of static analysis security testing is that it helps us detect vulnerabilities early before the code is deployed, merged or even committed. Uh this is especially powerful in fastmoving teams where issues can slip through t uh through cracks during code review or testing. Um unlike runtime tests, spoiler alert, static analysis uh doesn't require executing code, right? That's what makes it static versus dynamic. This means though that it can detect vulnerabilities in code paths that aren't covered by tests or are difficult to simulate. There's probably a part of your application that's really, really hard for you to set up in an integration test, but you might be able to catch some issues using static analysis.
So, let's walk through a few vulnerabilities that are especially relevant in Ruby applications. Um, and I I selected these because I think that they are easy they're easy to implement once you get to a certain scale, sort of that just above small sort of midsize rail shop. Um, where you find yourself bumping into things like the constraints of active record and you're like this would actually be much faster and more efficient um if I just drop down and write some SQL or maybe you take a half step and do aerial tables. Um, so they're kind of in this space. Um, and I [clears throat] I had some more examples that were kind of incorporated more like front-end code and I took them out and I wish I kept them in uh because it would have been a great segue um from Jeremy's presentation. Also, Jeremy, I did a quick scan of your code. Looks pretty good, man. Um, you there were um there's places in Vue component where people are
very very tempted to met a program and you resisted. Good job. Um, and I'm not going to tell Ruby programmers not to metapar program. It's just there's a time and a place. Um, mass assignment is not necessarily a side effect of meta programming though. Basically, this is what happens when user input is used to set multiple model attributes at once, often via params. Um, and so you see this, you know, when you're updating more than one model at once and maybe you have sort of a loosey goosey data object and you're not being strict with exactly what parameters you pass or allow list. Oh, and that's the other thing. You did allow listing on setting up the button classes, which is very important. So, all right, A+ for Jeremy. Everyone else, see me after class. [sighs and gasps] Just kidding. Um, so this would allow a user to modify fields that they shouldn't have access to, such as admin or user ID.
Um, this Ruby vulnerability cross- sight scripting um it does so it uses contact tags and content tag and uh using the Rails tags uh will actually escape um dangerous characters. So that's a good thing to do. Another point for Jeremy. Sorry, this is the Jeremy. You did a good job talk. That's the name of this talk. It's over. Um, so Rails provides cross-sight scripting protection by default through autoscaping. Um, but there are some dangerous methods out there that you can just call. Um, and two of them are HTML safe and raw. And there is maybe a little bit of a miscon misconception about HTML safe. It does not make something safe. Um, when you call HTML safe, you're saying, "Hey, I think it's safe." Just kind of like, "You can trust me man."
Um and raw does the same thing except for um it it I think it wraps HTML safe but it handles nil gracefully. Um whereas HTML safe will start a small fire for you. Um and it's it's a common mistake. People are like oh I marked it safe. It's safe. But cross-ite scripting can allow users or attackers to steal uh cookies session tokens and then ultimately perform actions as a user. And um if your application has users, you probably know users hate to be impersonated. They do not like that at all. Um [sighs] insecure deserialization. Um Ruby makes it easy to serialize and deserialize objects and it's all an object in Ruby, right? But that flexibility can be dangerous. Um, if you des serialize untrusted input using marshall load functions or similar methods, you risk arbitrary code execution. Um, so this is a YAML example and I will fully fess up like I work at the company that has we host the YAML
injection machine actions and we let you pass executable code to that YAML. So, you know, like let me take the gun out of your hand, but also here's this one. So be careful. Um, so I'll I'll I'll fully admit my part in that. Um, but you know, if you don't need to use YAML, use JSON, something safe like that. Uh, this is not a a talk about how to secure your code, although these tools do give some guidance. Um, SQL injection. This, uh, stays on the OASP top 10 list. I think we have an OASP megapan here. Yep, there he is. Um, and this becomes a risk like I said when you reach that sort of between small to medium and a bit medium larger uh that guy gets it.
Um, [clears throat] size of Rails application and you're like active record is not cutting it for me. I need to drop down a little bit lower. But maybe you've been writing active record for a long time and you're not sure why you're choosing the SQL primitives that you are and you do something like, "Oh, I just need to look up whatever." And you don't parameterize the query, which is just adding that little question mark. Um, and we see that when we push the limits of database helper libraries and we're forced to get creative.
All right, this is a fun one. This is a fun one because I I feel like you see this example and you're like, Kylie, no one would do that. It's been done. It's being done. Um, this is command injection, right? And it allows arbitrary system commands to be executed. Ruby has a couple ways we can execute commands, right? System, um, back ticks, exec. And [clears throat] honestly, I just please don't use send. Don't use send. Um I understand that that's not entirely feasible. If you must use send, if you must use system, if you must use exact, please allow list exactly what can be sent to them.
So is this okay? Ruby's dynamic nature and metarogramming make static analysis uniquely challenging. If you haven't noticed, um, convention over configuration, which is, you know, the drum beat of DHH and Rails, can obscure intent and the lack of type annotate annotations that make Ruby flexible and fun to use make it really hard to infer data flow. And Rails has made a lot of security improve improvements. But the things specifically that make Rails so flexible and fun are what make it difficult to secure and difficult to detect those security vulnerabilities.
So static analysis can identify issues that tests may miss um such as injection syncs, eval system send, unsafe data flow from user input to sensitive operations. Um and you know the view component example that I think of is it's very common to want to uh pull a component off of a page and then send that as hey now go render me something that has the same name. But what you're doing is sending an unsanitized value from a position that can be controlled by the user. Um, or we can also detect misconfigurations or secure defaults that you just wouldn't catch with, you know, standard Ruby integration testing.
And static analysis security testing complements traditional integration testing by providing a different lens. Um, it doesn't rely on execution or coverage. So, especially in dynamic languages like Ruby where we have a lot of classes that could be built on the fly at runtime, static analysis helps surface those hidden risks early because we look at data flow. Um, yeah. So, why SAS for Ruby? Um, a test should hopefully check that this renders correctly without input. Um, you might check to make sure it doesn't render the wrong thing, but what it won't catch is that this is an injection sync. Static analysis tools like breakman or open grab can flag this as a potential cross-sight scripting vulnerability because untrusted input params are passed directly into the sync render endline. Um, and it's being rendered directly without any sanitization.
or this one. This kind of vulnerability is hard to catch with tests because unless you're explicitly testing for malicious input in every single controller, you're not going to find this. And I mean, are you going to add a test to are you going to add a a condition to every single method test in every single controller? I don't know, maybe. But a tool like CodeQL or open GP with taint tracking can trace the flow from params Q to eval and flag it as unsafe. So SAS tools are usually comprised of one or more primit primitives to detect vulnerabilities.
I've divided the primitives that SAS tools use into three categories. Reax or regular expressions. And from Javier's talk, I know that a lot of you are familiar with reax um as or abstract syntax tree and data flow tracking or taint analysis. So, reax it is simple but pretty quick pattern matching, right? Um super fast but super brittle. Um so there you see we have an eval of a constant to reax. Eval looks dangerous, right? depending on how you write your reax tool or how the reax tool is written in the SAS tool that you choose. Um, and then at the very bottom that looks safe. It it that's not flagged at all. So, it's a false negative. So, very brittle, but it's very fast. So, [gasps] um, move quickly. You don't have time to get it all right. Abstract syntax tree.
Now, this is pretty interesting. I'm not showing um exactly what is highlighted because abstract syntax tree is more of a way that your code is organized before it starts to detect things. It parses the code into a tree structure and then matches on patterns structurally. So you can see here we have call expression receiver is the class. It's receiving the method where and then you can see the arguments that are passed in the string literal with interpolation and then hash hash access on a param called name. All right, this is the cool one. I think data flow tracking. Um, it tracks data from the source to sync. A source is any kind of input. Commonly the one that we're worried about in security is user controlled, but anyone can put anything bad anywhere. I found um the sync or the injection sync is the exploitable method. In this case, it is the wear method being used without a parameterized query.
And dataf flow tracking detects tainted data reaching sensitive operations. So that's important, right? It actually checks it looks for data that could be tainted so that you have fewer false positives. This is the most powerful detection method, but it's also the slowest by a lot. Um, OpenGP, which is a fork of a private product called Sim Grep, builds an abstract syntax tree of your code, and CodeQL, which is GitHub's product, goes a step further and builds an entire database, indexing your codebase. All of this annotation enables the data flow tracking, but obviously it's extremely timeconuming.
So, let's talk about some of the SAS tools that are available and compatible with open-source Ruby projects. Um, so these are some tools that you may already use and think, hey, aren't these enough? And I have to start with Sorbet. Um, so Sorbet is not really a SAS tool at all. But I will mention it. It's a typing system for Ruby. And types can provide some safety about the values that methods accept and return. But security value, any security value that you get is purely incidental. And it's pretty difficult to measure exactly um what security value you get from types um because something can be the right type but still be malicious.
Uh Rubocop it is abstract syntax tree aware. It's more focused towards quality but it does implement some security rules that you can add and you could also write your own security rules if you wanted. Um, Breakman Breakman is a true static analysis tool, but it's built with Rails in mind and the protections that Rails provides. So, you can run this on Ruby only projects by using the force option. And that's what I did for the demo. If you go to the demo repo or you look at the demo slides or the code in the demo slides, you'll see this. Um, but you run the risk of false negatives because the tool makes assumptions that you're implementing certain Rails patterns.
And then there are some tools in the larger landscape that aren't necessarily Ruby P goodness specific but are pretty powerful. Um, OpenGP is an open source fork of SimP. It's a semantic GPA SAS tool that analyzes the semantics or behavior of code instead of just syntax for complex detections. It has a very robust set of community rules, but you'll need to verify and maintain those. DevSK is a Microsoft tool that integrates tightly with idees to detect vulnerabilities at the time of writing code, but it doesn't really have a strong built-in Ruby rule set. So, you'd have to let it write a lot of those rules yourself.
ASCRAP is a kind of scrappy open-source solution that's very flexible and lightweight. Um, and I like it for those reasons, but you will need to write most of the Ruby rules yourself. Finally, CodeQL's GitHub semantic data flow tracking and it also does crossfile taint tracking. Um, so it's important to note these are all available for open source projects um and then for free and then um for enterprise you'd have to pay for code and open grep is the fork of sim which you could pay for for uh enterprise but I don't think they want you to just use it at your business for free.
So, we know a little bit about SAS primitives and what each tool does, but what does detection look like? Um, let's look at some SQL injection, right? And see see what each tool finds. So, what we're looking for is um the direct string interpolation, right? And this is um same method, same file. So, pretty straightforward. Um I think it's something that a lot of tools should be able to detect. Oh, but Rubocop does not detect this with the built-in rules. You could write a custom rule for Ruby Rubocop, but this is very much the as quick as we can set up. What can we get? Um, what it does detect though is a couple of um dangerous evaluation methods. It got eval and got YAML load. Good job, Rubocop.
Breakman does a lot better. It found the SQL injection and it found some command injection and it's capable of some taint tracking within the file but not between files. This is pretty good and if you're running a Rails project I think Bregman's a good solution but remember it's really oriented towards Rails projects so it will produce false negatives on projects that don't use Rails. Devkim I wanted to love you. Um, DevSkim catches the dangerous deserialization, but none of the SQL or command injection patterns. You would have to write those Ruby rules yourself.
ASTRAP is really, really fast, but it only catches direct patterns like reax or the structural patterns using abstract syntax tree detection, but it fails on all data flow analysis. All right, OpenGP. Uh, this probably looks uh similar to the breakman example to you. Um I will say it again. Breakman is a great solution for Rails but if you are writing any other Ruby project it will fall a little short. OpenGR can catch a whole lot more. SQL injection command injection data flow injection within a file. Um OpenGP did not find the cross file track tracking with the community rule set. Uh that is part of Simgret's paid offering.
Um, CodeQL detects all the vulnerabilities that other tools did using dataf flow tracking, um, tracing from the source to the sync. And bonus, um, it also tracks dataf flow cross file, which is especially handy if you are metrogramming. I know no one here would do that. And doing things like mapping web or API controllers to model or other database backed classes. Um, and I don't show this for all of them, but if you look at my demo repo, you'll see this. I included this serif output. This is what gets consumed. Um, if you integrate with an editor like VS code, and you want to uh, you know, like render these results like in the editor, it gets written in a a file format called SIF. That's also what um, GitHub's code scanning product uses to render these code scanning alerts. And uh it's a bit boring but you can see that the it does a code flow and it shows you the source is from the users
controller and then it shows you the location of the execution is in a different file and that file is models user RB. Um so just a quick what each tool sees or misses. SQL injection can be detected with precision with breakman open grap and codeql. Insecure d serialization is caught by all tools because it can be pattern matched. Breakman open grep and codeql can find data flow within a file. Only codeql can find the data cross file. Um sim is supposed to be able to do this too but I have not paid for sim grep to um assess that myself but it is in their product offering. So I guess what I I want you to take from this is that Rubocop and Breakman are great, but the data flow tracking is really important to detect the vulnerabilities that meta programming can create.
So from the last chart, you're probably sensing I have all the logic and reason needed to make my decision. But let's bring back the sensibility for a moment. things like developer happiness, especially around ease of setup, maintenance, and use matter. So, if you look across the board, it looks like Breakman and Rubicop seem like no-brainers for Ruby and Rails code due to the ease of use, low false positives, and performance. However, they don't detect as many vulnerabilities as tools like OpenGP and CodeQL. Um what you might also want to consider breakman is Rails specific. So if it is a Rails project, I think that's a great way to go. Um CodeQL and OpenG have the best built built-in Ruby support. Um CodeQL has a really robust default pack and extended security suite that you can opt into and you can write rules for any of those and make it much more powerful and custom to your needs.
Um so where to integrate? I would say that you should integrate at least one quick win tool and one deep analysis tool. Quick win tools are effective at the editor and gith hook stage. So um these are things that can run really really fast and um they don't have to be unless they're blocking. If they're blocking they really you know can't have any false positives but unless they're blocking they can be like a little bit more loose on the false positive space. You don't have to be as confident in those. Um, and then I think you should integrate at least one tool in the deep analysis stage. Um, I think integrating deep analysis at the CI/CD stage is a really good idea because it reaches often your CI/CD reaches developers when they're in a pull request stage and they're kind of more in a review mindset and they're more amendable to making a change. Um, I also, uh, working at a company that does a lot of code
scanning, I can tell you statistically at my company, developers who are presented with security findings and a pull request are much more likely to fix them at that stage than they are if they get like a weekly report of their vulnerabilities. But, um, tools like CodeQL and OpenGP can take a long time to run and you may not have tolerance for that in your deployment. and you may not have, you know, five or 10 minutes depending on the size of your codebase for that code to run. Um, and another fun thing about CodeQL because it builds a database of your codebase. Uh, to speed up code runs, we cache uh cache those databases and then you have to think about when's the right time to bust that cache.
Um, Whoops. Yeah. So I I really think that the most effective stage is like GitHook for the quick wins and then CI/CD pipeline for deep analysis. But you really have to think about how your team works and what they'll respond to. So you can see all of the code samples and some more analysis at the repo that I linked. Um and I used a really cool tool here. I didn't make the tool called ACT. Um and I'd encourage you to use it if you use GitHub for your CI/CD. It lets you test GitHub actions workflows locally. Um, so I set this up and the actions run in the open source repo, but if you pull it down and you follow the install steps, you can test the workflows locally. If you don't want to test them, you know, if you don't want to push the repo to test them. So, I recommend that you audit your SAS and make sure you're implying both sensibility uh quick wins that delight rubists and sense, the deeper, more intensive tools
to catch complex vulnerabilities. And I think that the way to balance this is to remember that the best static analysis security tool is the one that your team will actually use. This was SAS and sensibility. Thank you.
if we have time. Yeah. Questions. Anybody just have some code I can pull up and I'll tell you what's wrong. I don't need to run the scanner. Um, okay. The question is, the ones that don't have as many rules, um, could you find rules that have been written? Community rules. Yeah, I think for rubocop, yes, you can. For devk, not as much. And then ascrap, it really just is like one guy, which like we've had, you know, heard the pros and cons of you using the library um built by just one guy. I don't know, it's a scrappy little project. I love it. I I will say this I do think that this is a good use of large language models where if you have a rule in one rule set you can say hey I have a rule for open grap can you please write an asked gp for rule for it or I have a breakman rule can you write it into codeql for me please because I want to know if this is happening across files you you're going to hate this you have
actions for codeql like you are running codeql and a action you're and CodeQL on your actions. Okay. All right. So, you are probably running um default CodeQL setup. Um honestly, I think that's a really good strong start. If it is a Ruby project, I would recommend that you turn on the security extended suite. People don't love it because they feel like it's noisy and they feel like some of the stuff is defense and depth. Um working in security, I have strong feelings about defense and depth, but your GitHub account is extremely valuable. um everyone's is. So I think that it really doesn't hurt to take like that extra step of using the security extended suite.
Ah that's a really good question. I didn't have a ton of time to look into um the Ruby parsers. I would say what the Ruby parsers really help with that is not as present here as I'd like to see is um editor integrations and being able to um implement like asked aware scanning in the editor. So ask kind of does that but yeah a a Ruby parser that is strong I think would be really really good but I haven't played with that one myself. Sorry. Do I have any war stories from GitHub that saved a lot of people? Um, I think it's important work. I'm not sure that I've ever saved lives.
I do like that you think I've done that. Um, I've saved data. Um honestly I would say like the some of the most important work that we've done in this space has been um setting up the codeql rules and scanning for actions. Um you know we turned on actions people like it. I joked with someone you know the injection machine is full of injection because actions is the injection m machine right these YAML workflows can execute code in other languages and that's what we want them to do. Um so enabling the scanning on that and setting up custom rules especially things about um if there's one thing you want to do in your actions workflow pin to shaw on these dependencies right that you're calling u don't just call them at the tag because the tag uh the value at the tag can change under you and that's by design as a security engineer please pentaw um use minimum permissions for your actions tokens again not life- saving stuff but
kind of company saving stuff Um, yeah. So, I I mean I love actions. It's an amazing tool, but um because there's so much flexibility, it is inherently a lot of security risk. Um, but I could talk about action security all day. I get the impression Jim wants this event to end at some point though.
Questions about this episode
What is discussed in "XO Ruby Atlanta 2025 - SAST and Sensibility: A Rubyist’s Guide to Static... by Kylie Stradley"?
In order: 0:00 GitHub, Sense and Sensibility, Kylie Stradley; 10:00 Breakman, CodeQL; 15:00 CodeQL, GitHub, OpenGP, Breakman; 20:00 Breakman, CodeQL, OpenGP, Rubocop; 25:00 GitHub, CodeQL, Breakman, OpenGP; 30:00 CodeQL.
What is mentioned in this Confreaks episode?
12 people, works and ideas are mentioned, including CodeQL, GitHub, Breakman, OpenGP, Rails, Ruby. Each one on this page links to its own page with every episode that mentions it.
How long is "XO Ruby Atlanta 2025 - SAST and Sensibility: A Rubyist’s Guide to Static... by Kylie Stradley"?
The episode runs 32 min; the transcript is about 5,101 words in 39 timestamped sections.
More from Confreaks
About this transcript: the episode's spoken words as captioned on YouTube, split into timestamped sections. Reclip.Me locates every person, work and idea in the text, keeps the time it is first named, and quotes the words around it verbatim. Last updated January 24, 2026.