Hello Romain,

The Central Intelligence Agency would like you to consider a career with the National Clandestine Service. The CIA’s National Clandestine Service seeks qualified applicants to serve our country’s mission abroad. Our careers offer rewarding, fast-paced, and high impact challenges in intelligence collection on issues of critical importance to US national security. Applicants should possess a high degree of personal integrity, strong interpersonal skills, and good written and oral communication skills. We welcome applicants from various academic and professional backgrounds. Do you want to make a difference for your country? Are you ready for a challenge?

All applicants for National Clandestine Service positions must successfully undergo several personal interviews, medical and psychological exams, aptitude testing, a polygraph interview, and a background investigation. Following entry on duty, candidates will undergo extensive training. US citizenship required. An equal opportunity employer and a drug-free work force.

For more information and to apply, visit: www.cia.gov

You can make a world of difference.

Com'on guys, I'm not even US citizen... So yeah, CIA is looking for security guys by spamming on linkedin groups. Anything wrong in that process?

For the one that don't know Qt, this is a huge and mature framework for developing GUI & more on different platform (to read, multi-platform). I already did some development using Qt and C++ (especially when I was working at the GERAD).

As, with Marcin, we wanted to have a look at some technologies that involved a browser etc. I decided to look at Qt and the almost-fresh WebKit integration.

The integration of WebKit in a framework like Qt, allows the developer to embed supposedly in a easy manner a browser that supports the basic web technologies which are HTML, CSS and JavaScript (it seems that Flash is going to be supported soon, and anyway, one can write its own plugin in order to interact with some specific content) in its application.

And indeed it is easy... I used PyQt in order to develop a very simple prototype and see what we are able to do with this new technology. As I know already Python and Qt, it was easy to me to start and be kinda effective. So, in few hours of work, documentation reading and trying to understand why and how the Python version of Qt was using such or such thing compared to the C++ version, I got this workable browser that allows dynamic JavaScript injection through a console, view the source and a simple encoding converter (click on the image to see the full screen-shot):

At this point, I was actually very excited, less than 500 lines of Python in order to create that... was kinda worth few days of work in order to create a useful tool: the Swiss Army Knife of the Pen-Test.

My next and logic step was to extend the current tool in order to have the tamper-data like capabilities (eg. being able to hijack the HTTP request and then tampering the GET/POST data).

And here come the problems... it's apparently not possible to get the current request then reply when using the WebKit widget in Qt (QWebView). I tried to use a delegate QNetworkAccessManager in order to overload the POST/GET request since this object is use to set the proxies etc. but nothing... I think they just didn't open this possibility for some reason.

Oh well, I then stop developing this prototype and will try to contact Qt experts/developers just to figure out if there is no other way to do it. I thought of a solution which would be to have my own HTTP manager using QHttp in order to do the request, get the response etc. and then sending the content to the browser; this would be great in a webapps scanner, but for the use that I wanted with, that would create huge limitation for the user-interaction and especially for Ajax applications. So, the prototype stays here until I find a solution or Qt open their network management under the QWebView widget...

Fixed:

An update to let you know that I actually fixed the problem, it was really stupid from me, but I should really care when the method are virtual or not before overloading it or not :/ shame on me!

So now, I am able to have a firefox/tamper-data/firebug in one tool :)

While working on the C++ version of scalp, I had to do massive simple transformations of a given text, ie. replacements of words by others.

Since the main way to do this (a loop which does a replacement at the time), is very inefficient, I decided to find something faster. I then came up with a tree based replacement algorithm; I believe this is kinda famous but I never heard about such algorithm, it basically uses a non compact trie in order to have an efficient search of the current word.

The main algorithm is very simple and similar to a state machine where the state depends on the next character in the trie. For example, if we want to to replace the words: "ba", "me", "mp" in a text, the trie will be this following one:

The idea is then to iterate over all the characters in the text, and for each letter determines whether this is a possible word to replace or not (simply by looking if the letter is a child of the trie root). Then, we iterate over the next letters in the text in order to see if the sequence of letters are an actual word to replace or not (every time, the same methodology is used: look in the children at the current state of our iterator in the trie).

This algorithm seems more efficient than the simple replace used in a loop since we will perform a descent in a tree and therefore replace a linear search by a logarithm one.

I ran a little statistical comparison between two algorithms: mine and the simple loop one. The test bed is quite simple and uses randomly generated text which contains the words to replace with a certain density. In order to create statistics, I made all the sizes varying and I aggregated the results from the same dictionary size. So, for a given size of a dictionary (let's say, 200 words to replace), a text has been generated with a density that vary from 0.1 to 0.5 (from 10% to 50% of the words in the text will be words to replace) and finally, the size of the text vary from 25 to 200 words (and words are randomly generated to be from a size 5 to 32).
As I said previously, the results from a same dictionary size has been aggregated since I've seen practically that the result mainly depends on the dictionnary size (it also obviously depends on the size of the text, but as this is a constant for the 2 algorithm, I can compute the mean of the different data to extract the average gain for a particular dictionary size).

Finally, here is the curve that shows the logarithm progress of the gain compared to the classical method):

The reference replace implementation which has been compared to the one I developed is the following (STL/C++ implementation):

void str_replace(string& where, const string& what, const string& by) {
  for (string::size_type i  = where.find(what);
                                 i != string::npos;
                                 i  = where.find(what, i + by.size()))
    where.replace(i, what.size(), by);
}

Yesterday, I came across a case in a piece of software which was really hard for me to understand perfectly. Not only the code is well written (which is always worse for finding bugs :)) but the structure is also well thought (this is the implementation of an associated array in C in the lighttpd application).

The problem I had was to state whether a tool report was a true-positive/false-positive. So, as in many case I've seen in this software a problem may occur only in the limit cases. This one may occur after INT_MAX insertion in the structure. I don't know if one of you ever tried to do such a thing, but only INT_MAX (~2 billions on typical PC) allocations is a lot, so inserting elements in a structure that needs at least 5 (re)allocations is too much. But well, I did it. Also, I ran this test with valgrind using the memory leak check (full check and high definition).

I then ran a simple test program to fill this structure in a real condition: a typical x86/32-bit architecture. As I knew it was stupid and didn't even think this could end before 2 days I started looking in other direction in order to reduce the INT_MAX size for having a reasonable time execution of the test.

My first attempt is to shift all the types that are used, I knew this was not perfect because even if I can force my program to use unsigned short instead of size_t, I wouldn't change the size of the pointers, a char * would still b 32-bit (there may be some options in gcc to control the size of the pointers — which I doubt — but I didn't find any). Using this methodology, I was able to make the program crash in the way that would have been a real true-positive.

But as I knew it was not good since the size of the pointers are not modified and I had the feeling that in that particular structure, the case of the possible crash is handled by itself (due to pointer and type limits), I started looking in other direction for running that program in 16-bit, a pseudo-real-16-bit-mode. I then started looking into emulators and how to compile code for 16-bits and running it on my linux (x86/32-bit). After having issues compiling and running the test program with the gnu-m68hc11 ELF package, I found the bcc/elksemu stuff. After compiling and running with ELKS utilities, the test program didn't crash, it only failed in an assertion test after an allocation...

Different behavior, with different methods, okay... which is the correct one? Is it a problem of pointer size that made the test running differently than the real program on a 32-bit or maybe a limitation of the elksemu machine? As this morning I checked the state of the 32-bit run I launched yesterday, and this was finished... ended by a failed assertion.

As expected, pointer size matters when you wanna test on intrinsic limitations of a structure and its behavior using limit cases.

Since I started to work on my static analyzer using php-ast/oracle, I realized that looking for vulnerabilities need a lot of hard coded/database entries. This is really sad, since, in order to get something correct you would need a huge knowledge database. So I started thinking of generalization of vulnerabilities and way to express it. It's tough. Really.

The most realistic (if I can say so) idea I had is to actually handle vulnerabilities definition using a given taxonomy. I still need a lot of knowledge, especially on the language (PHP) I'm analyzing, especially the output functions, global variable, filters, resources etc. but the big advantage with rules is that you can generalize the definition.

Anyway, I started dealing with natural language, will try to make this fitting into my model in order to communicate with the future static analyzer engine of php-oracle... and thanks to the AIMA project, I was able to get some fast results on the processing:

# source definition:
unvalidated input go to sink in html context
# parse tree:
2 possiblities
##
  02NP[('Adjective', 'unvalidated'), ('Noun', 'input')][]
      23VP[('Verb', 'go')][]
        45NP[('Noun', 'sink')][]
       ('Preposition', 'to')
      35PP[]
     
    25VP[]
      68NP[('Name', 'html'), ('Noun', 'context')][]
     ('Preposition', 'in')
    58PP[]
   
  28VP[]

08S[]
##
  02NP[('Adjective', 'unvalidated'), ('Noun', 'input')][]
    23VP[('Verb', 'go')][]
        45NP[('Noun', 'sink')][]
          68NP[('Name', 'html'), ('Noun', 'context')][]
         ('Preposition', 'in')
        58PP[]
       
      48NP[]
     ('Preposition', 'to')
    38PP[]
   
  28VP[]
 
08S[]

And the taxonomy I used is the following (which needs to be extended to handle more than "input validation"):

IV = Grammar('InputValidation',
	Rules(
		S = 'NP VP | S Conjunction S',
		NP = 'Pronoun | Noun | Article Noun | Adjective Noun | NP PP | NP RelClause | Name Noun',
		VP = 'Verb | VP NP | VP Adjective | VP PP',
		PP = 'Preposition NP',
		RelClause = 'That VP'
	),
	Lexicon(
		Noun = "input | output | privilege | context | header | user | sink | file",
		Verb = "is | go | write | print",
		Adjective = "validated | unvalidated | asynchronous",
		Pronoun = "me | you | i | it",
		Name = "html | database | http | sql | ldap",
		Article = "the | a | an",
		Preposition = "to | in | on",
		Conjunction = "and | or | but | not",
		That = "that"
	))

Now, I only have to finish my model of a vulnerability (I do not think about building something really general, but a model that can handle injection flaws, privilege, communication would be awesome). Once this is finish, lots of things would be possible such as generating attacks directly from the definition (this would be more like a generalized attack generator) and vulns. checkers for the source code analyzer.

I know this is a kinda tough project and I really have lots of other things to do, but I really want to give this a try... just to see where it goes...

I used to work a bit on spam bot protection, whether it is for protecting the email disclosure or the spam in the website itself. I then, started a stupid website called http://fuckthespam.com where I will gather some spam (the funny one) but also listing some anti-spam techniques :)

Hopefully I will be able to also make an history of spam to see how techniques and also content evolved.

And I've just received this book this morning:

I love firebug, this is something really good for developing web apps. But today, I got an issue which was pretty annoying! First of all, when I develop a small apps, I used to do this under firefox only with firebug and other nice extension loaded.
But today I got an issue when I wanted to overload the XMLHttpRequest send function to do other things with: Firebug simply do not allow me to do this, but it works well if I want to overload the 'open' function!

Pretty annoying but you cannot do this with firebug activated:

XMLHttpRequest.prototype.send = function(data) {
    sData = transformation(data);
    this.originalSend(sData);
}

Always on the same subject: Spam bots, i was thinking that obfuscation would be a good way to prevent spam bots. Then I first start playing with reverse strings even if it may be obvious for the bots but well, I'm pretty sure it's even more difficult than the previous technique which can almost be passed with an intelligent-but-with-no-javascript-support parser.

So this version is quite simple:

<script>
String.prototype.reverse = function() { return this.split('').reverse().join(''); };
function reverseNames() {
	formElement = document.forms[0].elements;
	for(var i = 0; i < formElement.length; i++)	{
		formElement[i].name = formElement[i].name.reverse();
	}
	formElement.submit();
}
</script>
...
<form method="post" action="check.php" onsubmit="reverseNames()">
	<label for="emanresu">&#8238;emanresu&#8237;</label> <input type="text" name="emanresu" />   <br />

You can find the running example: here.
While talking about obfuscation/crypto, since there are few parameters to obfuscate/encrypt maybe a Vigenere algorithm would be nice...

Note that we do not use the 'username' instance in the HTML page, if you want to print 'username' you can use the character ‮ which reverses the following text.

This a really nothing to do with web application security, but a friend asked me how to protect a bunch of html files in a directory. He was looking for sessions based solutions but for this he would have to rename the html files in php or whatever and then, implement the protection... pretty boring!
I suggest him a really easy and not perfect solution: checking the referer when accessing the html files (this is the kind of protection as the images anti-thief):

# .htaccess
# -------
RewriteEngine on
RewriteCond %{HTTP_REFERER} !.*yousite.com/.* [NC]
RewriteRule ^(.*)$ /fail.html [NC]

You can find an example here.

PS: this could not be a valid solution for lots of application!

The previous article on spam bots was focus on using generation of the document itself by adding dynamic content (JavaScript) or a random fields... By the way, Jungsonn follows with a clever technique of the same kind.

Today, I was looking at the logs of a phpBB forum we have in an association where we have lots of spam; okay phpBB is famous and maybe not the most secure but well... there is some prevention (modules such as CAPTCHA) for this and the webmaster installed lots of them...
I was quickly able detect manually the bots actually, this is something like same processes even if they introduced lots of variations and of course a detection system cannot be time based because they prevent this.
So we can easily see that it's tricky to automatically detect a bot without learning machine, SVM etc. but it's not my point here.

If I cannot easily prevent them coming on the website or track them, I can at least manipulate in some way:
Fake login page which cannot be seen by a user (<a href="login.php?sid=THE SESSION ID" style="display:none">Login</a>; As display:none can be easily detected by the bots we can trick this with the JavaScript "expression" evaluation in CSS...) and in the fake page adding some information in the cookie/session such as SESSION['BOT'] = 'DETECTED' etc.
We also need a verification script embedded in every page which would do generate a blank page or something like that... Don't forget also to ban the IP address.

I will try to develop this idea and test it because now, I have no proof that it can work properly. Furthermore, it also need the bots not to support CSS a/o JavaScript.