Comments on: More on Web App Scanners http://anautonomouszone.com/blog/archives/22 An autonomous zone to promote an exchange of ideas, skills, and experiences with computer (in)security. Tue, 15 Sep 2009 22:49:34 -0600 http://wordpress.org/?v=2.8.4 hourly 1 By: Chuck B. http://anautonomouszone.com/blog/archives/22/comment-page-1#comment-14 Chuck B. Mon, 14 Jul 2008 22:29:31 +0000 http://anautonomouszone.com/blog/?p=22#comment-14 Caleb, I do appreciate you taking the time to comment on my post. Your comments are generally thoughtful, insightful and honestly just plain better than most that I get (not that I really get a lot at the moment). However, I don't think you've corrected me so far. When I said “the above technique is what 99% of web app scanners do when looking for XSS” you commented that that's not how your product currently looks for XSS. I never said that's how your particular product did work, I never singled out webinspect and said "this is how they do it" - apparently your product is in the minority. That being said - there are some scanners that do a lot more complicated things to detect vulnerabilities (like XSS) than I described. It doesn't always mean that it makes them much better at actually finding vulnerabilities. Some of the "more complicated" scanning techniques sound really good in theory. From what I've played with and with in-house testing (as described in my NTOSpider post) the end results a scanner produces just don't always match up with cost, complication or bloat. And that is what got me going on this topic - just that I can't seem to find anything (or hear of anything from someone else) that works that much better than what I described above in a real-world comparison. They all sound good - but they don't all work good. Finally, I'd like to say that I sincerely hope that someone does come up with something that works really well and blows everyone out of the water - and makes finding vulnerabilities a ton easier. And not that I agree with that crack-pot comparison by Larry Suto, but web inspect didn't necessarily blow the competition out of the water in any way - and I've never heard of a third party comparison test where it did. From what I can tell from working with a lot (personally just hundreds, not thousands) of web apps from junk to enterprise, IMHO the big boy expensive scanners still have a long way to go before they can justify their cost. Cheers, Chuck B. Caleb,

I do appreciate you taking the time to comment on my post. Your comments are generally thoughtful, insightful and honestly just plain better than most that I get (not that I really get a lot at the moment).

However, I don’t think you’ve corrected me so far. When I said “the above technique is what 99% of web app scanners do when looking for XSS” you commented that that’s not how your product currently looks for XSS. I never said that’s how your particular product did work, I never singled out webinspect and said “this is how they do it” – apparently your product is in the minority.

That being said – there are some scanners that do a lot more complicated things to detect vulnerabilities (like XSS) than I described. It doesn’t always mean that it makes them much better at actually finding vulnerabilities.

Some of the “more complicated” scanning techniques sound really good in theory. From what I’ve played with and with in-house testing (as described in my NTOSpider post) the end results a scanner produces just don’t always match up with cost, complication or bloat.

And that is what got me going on this topic – just that I can’t seem to find anything (or hear of anything from someone else) that works that much better than what I described above in a real-world comparison.

They all sound good – but they don’t all work good.

Finally, I’d like to say that I sincerely hope that someone does come up with something that works really well and blows everyone out of the water – and makes finding vulnerabilities a ton easier.

And not that I agree with that crack-pot comparison by Larry Suto, but web inspect didn’t necessarily blow the competition out of the water in any way – and I’ve never heard of a third party comparison test where it did.

From what I can tell from working with a lot (personally just hundreds, not thousands) of web apps from junk to enterprise, IMHO the big boy expensive scanners still have a long way to go before they can justify their cost.

Cheers,

Chuck B.

]]> By: Caleb Sima http://anautonomouszone.com/blog/archives/22/comment-page-1#comment-12 Caleb Sima Mon, 14 Jul 2008 19:55:55 +0000 http://anautonomouszone.com/blog/?p=22#comment-12 First off I agree 100% on that scanners are not meant to replace manual testing and honestly I'm not sure where people got that idea in the first place. In fact webinspect was born because I was tired of munging around with my perl scripts when doing assessments. I needed a tool to help me do things quicker.. thus the product. A lot of people are quick to judge on the product not finding session hijacking or detecting logic errors but I say to them we do our best to automate as much as possible and make it easier for YOU to do your job but the rest is up to you.. besides thats what they pay you for :) I would like to comment back on what "99% of web app scanners do when looking for XSS". I realize this is the most obvious way to detect for XSS but Webinspect and Appscan(I'm pretty sure) stopped using regex's (as much as I love em) about 4 years ago. Today our xss detection method is slightly more advanced. For instance step 1 involves sending probe queries to determine where our attacks are being placed in the DOM, are we in an href? javascript? css? etc.. Once we identify where we are we then craft our attack based on that so that we escape out of anything that we need to in order to get proper execution. (we also use to identify what characters were being filtered here so that we crafted our attack only using allowed chars but we removed that due to false negatives). Once we have our attack we send it in and then actually evaluate the response in order to determine execution. So no regex parsing. We load the script and execute the page and if our script executes.. bam! XSS. Now this is a much simplified explanation and this is only for vanilla reflective XSS.. we have not even delved into stored XSS and how complicated that can be for an automated product. Its humorous to me how many people really don't realize how complicated these things get. Web app scanning is definitely a world where experience is the key. You may find that a perl script helps you on sites that you test on but if we were to put it in front of our thousands of customers with web apps that range from junk to enterprise.. you find out very quickly how much is missing. Keep up the good postings.. hope you don't mind me throwing in a little correction now and then :) First off I agree 100% on that scanners are not meant to replace manual testing and honestly I’m not sure where people got that idea in the first place. In fact webinspect was born because I was tired of munging around with my perl scripts when doing assessments. I needed a tool to help me do things quicker.. thus the product.
A lot of people are quick to judge on the product not finding session hijacking or detecting logic errors but I say to them we do our best to automate as much as possible and make it easier for YOU to do your job but the rest is up to you.. besides thats what they pay you for :)
I would like to comment back on what “99% of web app scanners do when looking for XSS”.
I realize this is the most obvious way to detect for XSS but Webinspect and Appscan(I’m pretty sure) stopped using regex’s (as much as I love em) about 4 years ago. Today our xss detection method is slightly more advanced. For instance step 1 involves sending probe queries to determine where our attacks are being placed in the DOM, are we in an href? javascript? css? etc.. Once we identify where we are we then craft our attack based on that so that we escape out of anything that we need to in order to get proper execution. (we also use to identify what characters were being filtered here so that we crafted our attack only using allowed chars but we removed that due to false negatives). Once we have our attack we send it in and then actually evaluate the response in order to determine execution. So no regex parsing. We load the script and execute the page and if our script executes.. bam! XSS. Now this is a much simplified explanation and this is only for vanilla reflective XSS.. we have not even delved into stored XSS and how complicated that can be for an automated product.
Its humorous to me how many people really don’t realize how complicated these things get. Web app scanning is definitely a world where experience is the key. You may find that a perl script helps you on sites that you test on but if we were to put it in front of our thousands of customers with web apps that range from junk to enterprise.. you find out very quickly how much is missing.
Keep up the good postings.. hope you don’t mind me throwing in a little correction now and then :)

]]>