r/sre
One base image change produced 340 findings across 200 accounts and my priority list stopped meaning anything, then we fixed the image and the findings stayed anyway
- upvotes
- 1
- comments
- 8
Post
Highlighted: the lines this signal was extracted from
Our cloud priority list had 340 criticals on it last month and every one traced back to the same base image, which i reported upward before i understood what i was looking at. Two hundred accounts, roughly 340 instances built off the same AMI, so one mistake gets replicated at whatever scale your estate is at and it looks like a catastrophe instead of an afternoons work. Sort by severity and you get the same row 340 times. I tried deduplicating on title which fell apart the moment two titles differed by a word, grouping by account which lost the real total, and suppressing duplicates which took some real ones with them. Whats actually missing is a parent. The unit of work here is the image, not the instance, and no console i have seen wants to give me that. we have all 200 accounts in axonius next to the scanner output, and it shows the same 340 rows until somebody tells it to group on the image, which is where im at now. The scanner isnt wrong either, the estate really does have 340 instances with that misconfiguration, its the unit thats broken not the tool. Fixing the image was one change. The 340 findings stayed, because the instances were already built and running by then. How are you parenting cloud findings so one mistake reads as one thing.