I <3 Bots!
Subscribe to the RSS feed

Keyword - Hot stuffs

Entries feed - Comments feed

Wednesday, August 29 2007

I now understand why it's difficult!

Okay, I know for the halting problem etc. Some theoretical stuff... But now that I'm working on one, I have to say:

Damn! That so complicated to do a source code scanner!

The dataflow is a real pain in the ass, and we know that it's impossible to have a real and full dataflow. But well, we need to do some. The dataflow is more complicated theoretically but what about the control flow? No really easier! I mean... that's easier but there are so many things to understand, so many patterns to recognize in order to build the model of the source code... And I'm not even talking about inter procedural stuff, multi-file source code etc.

So, I'd like to apologize to "I don't remember who are these people" but some source code scanners are good :) Well... for the moment! I'm really waiting for to see more high-tech stuff and AI in these kind of programs...

Anyway, I'm currently building a core engine working on a AST tree generated by yaxx (XML version). I have two short terms targets:

  • Real Obfuscation (from one source code to an equivalent with a different control flow... yes, not only rename the variables, functions, classes etc.)
  • A variable tracer (tool for pen-tester: $_GET['foo'] -> ($foo <- htmlentities()) -> echo or this kind of stack...)

Monday, July 23 2007

Python script utility called wwwCall and Grabber news

wwwCall: HTTP(S) utilities

wwwCall is a very small module for Python (tested under python 2.5 but should be okay for python >= 2.3) which handle the HTTP(S) connection with some special features like proxy, cookies, authentification (basic, digest). This morning, I was working on Grabber and I just realized how ugly the code was, mostly because of how I handled the web connections, so I decided to create a simple module to do the job easily. The idea is to have a single object handling some basic function of the python urllib2.

If you have ever use Python for doing web calls, you'll see that the utilization is damn simple and I think, pretty cool... Example:

# create the object
http = wwwCall('http://rgaucher.info')
# add the features you want (cookies,auth)
http.setCookieFile('./the_path/file.cookie')
# reaching a logging URL and saving the cookie
http.post("http://rgaucher.info/login.php",{'username' : 'foo', 'password' : 'bar'})
# register the username/password for the basic authentification
http.setAuthBasic("romain","mypassword")
# print the content of the protected page
print http.get("http://rgaucher.info/401protected").read()

Download: wwwCall.zip

The next Grabber

So, I've been working on Grabber for a couple of months without a release now; it's mainly because I don't have that much time to work on it, but also because I made lots of modification. Today I added a couple of features:

  • Understanding some mod_rewrite rules for the spider
  • URL exclusion
  • Basic/Digest Authentification

This comes in addition on the previous features I added, mainly:

  • Multi Site
  • Multi threads
  • Cookie analyzer
  • XSS Locator in addition of the XSS Fuzzer which is definitely faster
  • Spider module, only to crawl the site and export it in XML
  • Login ability, keeping session state

I cannot give a d-day for the release of the 0.2 version because I really want to have a more stable product and will feed some test suites I made at work the tool, to be sure it's reasonable (I will not give comparison results with commercial products :P). I also want to have a better spider...

Tuesday, July 10 2007

Website functionalities coverage

Coverage is a tool written in Python which allows you to track what functionalities/web pages are reached on your website. I use this tool for in my Web Apps Scanner evaluation methodology in order to know if the web apps scanner was able to scan every pages, every functionalities of my test apps.

Anyway, this tool is pretty easy to use even if it requires a MySQL database to store the EntryPoints of the application. Basically, you setup the database, you insert the entry points into your code and you run the python script which will generate an HTML report with SVG graphs, reporting the coverage of your application.

Here is a report example

Installation

1/ Database

The database design I used for storing the needed information is the following:

CREATE TABLE `coverage` (
`CoverageID` int(32) NOT NULL auto_increment,
`Apps` varchar(128) character set utf8 collate utf8_unicode_ci NOT NULL,
`Date` date NOT NULL,
`EntryPoint` varchar(255) character set utf8 collate utf8_unicode_ci NOT NULL,
`Origin` varchar(255) character set utf8 collate utf8_unicode_ci NOT NULL,
PRIMARY KEY  (`CoverageID`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=1;
  • Apps: name of the covered application
  • Date: time when the entry point is reached
  • EntryPoint: Name of the entry point with a special format:


** File Reached:
Touch_ + Name of the file with extension, example, Touch_Index.Php, Touch_Search.Php etc.

** Functionality Reached:
Name of the functionality + _ + Name of the file with extension, example, this sequence of entry points of the page Login.php of a given application:

  1. Touch_Login.Php : Enter the page Login.Php
  2. Username_Password_Login.Php : The username and the password are feed
  3. Call_Function_Login.Php : Call the function login()
  4. Call_Function_Succeed_Login.Php : The function login succeed
  5. Call_Function_Error_Login.Php : The function login reported an error
  • Origin: the origin string is the concatenation of the md5 of the HTTP_USER_AGENT a pipe and the date; this ID + date is used to be sure to study the same user.
<?php
// ...
$origin = md5($_SERVER['HTTP_USER_AGENT']). '|' . date("j-m-y H:i");
?>

2/ In the code

So, you will need to add, in your apps code, lots of entry points. I made a PHP source code to do that more easily:

<?php
class Coverage{
 private $coverage_id = false;
 private $coverage = null;
 function __construct() {
  $this->coverage_id = true;
  $this->coverage = mysql_connect('192.168.1.3:3306', 'test', 'test');
  mysql_select_db("test_collect");
 }
 function send($entryPoint){
  if ($this->coverage) {
   $origin = "";
   $origin .= md5($_SERVER['HTTP_USER_AGENT']);
   $origin .= ('|' . date("j-m-y H:i"));
   $entryPoint = mysql_real_escape_string($entryPoint);
   mysql_query("INSERT INTO coverage VALUES(NULL,'BankApp',NOW(),'$entryPoint','$origin')");
  }
 }
};
	
$coverage = new Coverage();
function register_EntryPoint($entryPoint) {
 global $coverage, $supportCodeCoverage;
 if ($supportCodeCoverage) {
  $coverage->send($entryPoint);
 }
}
?>

Insert this code in a header or something and call:

register_EntryPoint('Touch_MyFile.Php');

etc. in your code where you have functional difference.

Run the tool

To run the tool, you need to have:

  • Python + MySQLdb (the python MySQL API)
  • The date (in SQL format) you want to cover; for now, it's only one day
  • The Origin ID of the user (the MD5(HTTP_USER_AGENT)), basically, you will look at this in the database, or get it by your code etc.


example:

$ python coverage.py 2007-06-28 41942da0293d0b8afcfab4c2d10c2401
$ python coverage.py 2007-04-12

The script must be in the same directory of your files for now... you can download the archive here: coverage.zip

Wednesday, June 27 2007

A new web site: Planet-Websecurity

This is a really nice initiative from Christian and Ronald: http://planet-websecurity.org/
This is for now an aggregator for a couple of web security websites (really good ones). This site will replace 7 rss I already have :)

Thanks guys

I <3 Bots!