A Tester can be the best attacker

A defender has to think of thousand ways, an attacker has to think of just one.

First job, first experience and the new hires group I am part of, decides to have a Hackathon for the new hires. A Hackathon that is true to its literal sense. I said “Why not? Let’s do it.” I always wanted to be on the other side of the event though.

I thought to myself if I’m going to conduct a Hackathon, why not conduct one that is Microsoft wide. Truth dawned and I realized that I should stop kidding. New into the company and you want to test the mettle of the ones who have been in the system for this long, that’s not just bold, that’s probably many levels ahead of that.

Voila! The bunch of folks who took this project actually wanted it to be at least Microsoft IT wide. The gang-leader still tried to keep it low profile until we have something substantial, but a Hackathon, which was MSIT wide was definitely on the cards. I was like a kid who has been told that you have been given the responsibility to test the mettle of the entire world.

We brainstormed a lot for hours and then, started the marathon coding sessions. I thought that we will do this and that and see how it pans out. But by the end of the day, it was pretty clear that I am still no one who can pull this event off. All I had was the lamest SQL hack that anyone can embed in an application and the simplest form of application that can be reverse engineered (which I later realized that I wasn’t supposed to code it anyway as that was part of another hacking event). And on that, another epic moment dawned on me, how are we going to identify that someone was hacked and who was that someone’s hacker.

Xkcd - 327

Bummer! All for so much enthusiasm. The gang-leader was still very much optimistic and ecstatic about whatever the day had delivered. I wanted to join him but all I could think of was either this guy is actually a pro or we are just destined to doom. We left at 2 that night. And to be honest, I had no clue what exactly did we do that took us so much time.

Taking a step back, one thing which was pretty clear in our mind was that there would be two different hacking events which would go on in parallel. The one that I was working on, was where every team would be given a server of their own which was running on a VM. A web form application, which had bugs, was deployed on these servers (VMs) with an extra catch that everything including the Application Pools were stopped. The other event consisted of a hackable central server. The latter event was more like a game where difficulty level increases with every difficulty one just surpasses.

Then began our next night-out-coding session. There were a few informal hangouts about how to go about completing this project, but this probably was the only proper coding session after the first night. There was something different about this session though. The moment I entered the conference room where we all were meeting, I was greeted with this giant and beautiful architecture of the whole system. God, that was big! The first thing that popped in my mind was who can actually even imagine drawing that. No perks for guessing, it was the gang-leader. He went on to explain how he envisions the whole architecture. The architecture wasn’t complicated per se. In fact, if I were to draw one such, I would also have drawn one which was something closer in resemblance. But the point is – It was just big… insanely big.

that big

Anyways, we started to code. I picked my designing part of the client side server. In our informal hangouts, we had discussed a few vulnerabilities that we can use and expose for exploitation. Still unclear about how to identify the hacker and who hacked whom, I picked on six or so vulnerabilities and started working on them.

A hacked webpage would lead the server script to crash. In other words, an exception would be thrown. It was this Error 500 that we relied on to identify the hack. A custom error page for Error 500 was designed. When a page crashed, it invoked this page. This error page, in turn looked for a shared folder where flags (GUIDs, basically) mapped to the corresponding user’s crashed page were stored. This GUID was displayed to the attacker which he had to submit to us so that we can identify the attacker and the one who was attacked. The folder was refreshed with newer flags to maintain extra cautiousness against a “friendly-play” (Friends generally follow the “tu bhai hai naa!” (Come on man! You’re my bro.) strategy. They’ll ask their losing friends to share their flags. The already losing friend thus decides that they are losing anyways, why not let your best friend win.)

500

Though this flag generation and distribution module might appear small, it was the most critical module in the whole architecture as it was the only way we could identify someone who did the attack and the guy who was attacked. To keep it totally aloof from any crash, it was again divided into two separate modules – one module just cared about the flag generation and the other one just cared about the distribution process. This helped in avoiding deadlock conditions and conditions where a lock could be on hold for long when the files were queued for multiple read operations. Any key assigned that was older than the currently generated one by a level of 2 became invalid. Further these were developed as Windows services which started on boot to keep it hidden from users’ view.

After 3-4 such marathons, we were ready with everything. Everything… Sigh! So much for the hullabaloo. Then the testing season started. And the initial test phases passed off with flying colors. I was actually ecstatic. We syspreped the VHDs for client and server thinking that everything that was supposed to be done is done and we were ready for the word Go! But, thanks for one curious fella who still wasn’t satisfied.

We sat one night together to test it out from head to toe in totally isolated conditions with none of our credentials involved. And then every horror that we could have imagined came to life. The feeling could be described in two simple words – Nothing worked.

Bawaseer-1

The first thing that we found was, while creating the application, I left my credentials involved somewhere in it. So, when the system started in total isolation (as an administrator), the application looked for my credentials (which never existed in the first place on the server) instead of kicking off things as an administrator. Trying to debug it piece-by-piece became a pain in the butt. Realizing that it probably would become much more tedious, I created everything from scratch without using my credentials.

The next thing we found was, no matter how current the flags were and howsoever valid they happen to be, the central server said the flag being submitted by the capturer is invalid. We banged our heads to figure out the issue for nearly more than 3 hours but to no avail. With no plausible visible issue and out of frustration, I started counting the number of characters in the GUIDs (on someone’s suggestion, can’t recall who). Much Ado About Nothing. It turns out that flags being generated and the ones being distributed differed by that last always eluding character. But hey, anything that fixes your pending bug is soothing.

Then it turns out that a person can hack himself. That probably could have been the worst thing to have happened. All everyone had to do was submit all his flags when he failed in his endeavours to hack others. Yes, he would lose flags for submitting his own flag, but then he would get the points for at least submitting the flags, won’t he?

This whole thing was already blowing up in our face. What if something like this happens when the event is actually on? I can’t even fathom the consequences. Anyhow we held our senses instead of going into panic mode and carried on with our work.

blow up

This time to be extra cautious, we created multiple fake participants and started the game again. Surprise, surprise! It blew up again. The culprit this time again were the bloody flags. Though it may not appear as a big issue, it was very subtle. The distribution of the flags was pathetically slow. Be it a network issue or a processing power one, this should not happen. What if the flag expires before it is actually distributed by the distributing service. The flag owner will go on to be the best defender without even touching his system. Small fix, but it definitely needed one.

All it needs is that one last kick to make you feel that you just can’t do it. Feeling that somehow everything is working as it is supposed to work, I was beginning to feel now that maybe, just maybe, we can pull this off. But there has to be that one last thing. The last night before the event, we again tested the whole scenario from end to end. The issue this time couldn’t have been subtler. We realized that after about half an hour or so, everything just about came to a freeze. Something was eating up the whole memory. We always felt this but we never paid any heed to it, given the kind of issues that kept coming up. Even after a reboot, the same thing happened. Looking up the memory usage, it was clear that the utilization shot up just after a few minutes. It was just that it was almost a complete freeze which happened after that long duration. The issue this time was with an exploit we wanted the gamers to explore. But as the general bugs are, the developers don’t have any clue about it.

I, for one, actually felt as if I would get lost into the code just by looking at it. The code was just perfect, at least it appeared to be. Think man, you just don’t have that spirit. Which coding principles did you evade when coding which led to this? What could have been that blunder?!

Will work

Logs, they always come in handy! It is one thing which differentiates between a good and an awesome programmer. One doesn’t understand the essence of logs unless he experiences it firsthand. Always leave a trail somewhere so that you can think through your mess. The trail is your guide to improvement. When staring at the code didn’t work, we sorted to logging. A dummy log was created to look into the issue and bam, there you have your fix! The issue this time was an open port which was left opened in the memory and was never closed. But ports? I didn’t do any port.open() thingy. And here fellas, you realize another truth, “Why one should not rely on garbage collection!” There was a disposable object which was created and was supposed to be disposed which opened this port. So, we had to dispose it manually. Tired of all the staring, we syspreped for the upteempth time and then left with a sad face. Maybe this was just not supposed to be.

So what’s the point of this all? Why this big write up? Developers just write the code and think that they have won the battle. All it took was a few night outs to realize what a failure it can be if it were only the developers who drive this tech industry. Testers know their way and man, they know it well! They know how to break and what will make the system break. They are the dudes!

And as for the event, it went kickass! :)

Downloading the web folder

People generally ask “how can we download the whole web-server?” They keep looking for different softwares to do this but they forget that they already have one, inbuilt (generally) in their very own Linux system. All they have to do is to run

wget -H -r –level=1 -k -p <http://domain_name/address_of_the_folder&gt;

to download the folder. You can change your level of recursion for download by changing the value of level according to your needs. That’s it! You have your web folder ready with you.

The Must Have in Linux

Everytime I install a distribution of Linux (generally Fedora) on someone’s system, the first question that the owner asks me after the installation is what other software he may need apart from the ones already installed. Well, this post is more about those extra applications. This post covers almost everything that one may need. If you feel, I have missed out on something, you can add a comment and I will add that up in this list.

Autoplus+ 

I start off with this simple script that will help you get rid of most of your headache. Flash, Google Earth, Skype, audio-video codecs, VirtualBox, Imagination, DropBox – It installs just about all the daily usage things. The only catch is it works only for Fedora.

Chrome

The browser that you just cannot miss on is Chrome. Yes, Firefox is already there on most of the distros but you just cannot miss this one.

Guake/Yakuake

A drop-down console that keeps your terminal on your fingertips. While Guake is meant for Gnome, Yakuake is meant for KDE.

(VideLAN)Vlc

It’s like the list is never complete if you don’t see VLC there. This amazing open-source player is almost on all the systems be it a Mac, Windows, Linux or any other UNIX implementation.

Amarok (with gnome plugins)

Talk about songs, one just can’t forget Amarok. Though intended for KDE desktops, it works equally well on Gnome. The lack of multimedia key support in Gnome can also be done away with the gnome multimedia keys script.

Xchm

Again Okular is there to support the .chm files, but it comes no way near xchm. Try it to to see the difference.

XBMC

XBMC media center is another open-source media hub for the TV experience on your laptop. The skins are so beautiful that you will definitely fall in love with it.

VirtualBox

This freely available virtualizer is a must for any geek (be a tech or non-tech). Try new operating system or run a Windows software, you will definitely find this handy.

VMware

Another virtualizer but with extra command over hardware which thus leads to higher data transfer speed. Even the network(ing) and the network configuration are also easier.

Qemu

Qemu is both an open-source emulator and virtualizer. If you are a tech geek, you must give this a try.

Wine

Wine lets you run your Windows applications straight on your Linux. Though not an emulator in strict sense as it does not emulate each processor instruction as any other emulator would, it provides the software libraries which Windows software may require during installation. It still is under active development.

Unrar 

This will add the rar codec for extracting the rar archives.

CbrPager/Comical 

A comic book geek? Well, these cbr and cbz readers are definitely for you.

JDownloader

If you are a heavy internet user who downloads stuff from rapidshare, mediafire, hotfile or any such file-sharing sites everyday, then this download manager will be a boon for your daily dose.

GoldenDict

As name says, it is a freely available dictionary which will always be running in the notification area for your help.

NetBeans

This is a fully-fledged IDE, completely written in Java. PHP, Java, C/C++, Groovy or Ruby – you can do your development with this IDE.

HandBrake

HandBrake is an open-source media converter with a clean and simple interface. You can also convert your media files to the mkv container format.

GParted

GParted is a partitioning software to create, resize, move, delete, format and reformat your partitions. It can also format a partition with NTFS file system.

RecordMyDesktop

RecordMyDesktop is a desktop session recorder which is both easy to use and configure. It comes in both command-line and GUI mode.

TrueCrypt

A powerful encryption software that can be used to create on-the-fly encrypted volumes and partitions/drives.

Super Grub Disk

This will come in real handy when you are in need of a System rescue. It will help in restoring your boot loader.

And as a download manager don’t forget to add DownThemAll to your firefox.

One more thing that I would like to add is that, while configuring your ppp modem, the libusb1-devel rpm/deb file is generally missing on your system. So, don’t forget to install that before you start configuring your ppp modem.

I also felt like including the softwares like GIMP, Brasero, ffmpeg, Totem, OpenOffice/LibreOffice, Transmission/KTorrent and nmap. But they generally come bundled with almost all the distributions. Well, you may also enlist your recommendations in the comments if I have missed on a few. I will re-update this list accordingly. :)

The annoying ‘yum Error’

One of the most annoying error that I have faced on Fedora is the yum error:

Error: Cannot retrieve repository metadata (repomd.xml) for repository: fedora. Please verify its path and try again

After a lot of googling and going through forums, I have made a list of solutions that can fix this problem.

-> Sometimes, you may face this error just after a fresh install. It can be fixed then by
yum clean all
yum clean metadata
yum clean dbcache

It can also be helpful even in the cases when yum was working just a few hours back and suddenly the problem rose and you have no idea why.

-> One of the most common fix is editing the fedora repo file. The fix is uncomment the baseurl line and comment the mirrorlist and then edit the /etc/hosts file adding

80.239.156.215          mirrors.fedoraproject.org
213.129.242.84          mirrors.rpmfusion.org

Well it’s is the most common fix. But, it has never helped me.

-> Another fix is disable the repo that causes this erro and then do the yum update. This was what I found in some forums but none of the solution-seekers were satisfied. One can though try this. It might be just as helpful.

-> Sometimes you may need to fix the rpm db. Type
rpm -vv –initdb
If one still gets an error, he can further do
rpm -f /var/lib/rpm/__db*
rpmdb -vv –rebuild

-> If behind a proxy, one may have forgotten to export the proxy settings. He can do so by
export HTTP_PROXY=http://username:password@IP:port
export FTP_PROXY=http://username:password@IP:port
For permanent solution to text internet, one can create a proxy.[c]sh file in /etc/profile.d/ and type
export HTTP_PROXY=http://username:password@IP:port
export FTP_PROXY=http://username:password@IP:port
export http_proxy=http://username:password@IP:port
export ftp_proxy=http://username:password@IP:port

and then log out and log in.

And for yum, add
proxy=http://username:password@IP:port
to /etc/yum.conf and do the update.

Compiling Hadoop codes

After wandering for around a month and a half and pulling my hair off my head, it feels good when the work starts heading in a definite direction. Understanding how hadoop works and then start coding for it are miles apart.

One of the problems that I faced in coding was I couldn’t compile any of my hadoop codes that I wrote, not even the one that were given in the books. The error that came up looked something like-

xyz.java:5: package org.apache.hadoop.fs does not exist

import org.apache.hadoop.fs.Path;

^

xyz.java:6: package org.apache.hadoop.io does not exist

import org.apache.hadoop.io.*;

^

and so on..

The basic problem is the classpath. We need to set the classpath to compile our codes because hadoop library files are yet to be integrated so that they can be referred during compilation. This can be done by-

$ javac -classpath hadoop-common-0.21.0.jar <filenam.java>

you can add -verbose option to the command-line so that you can actually see what’s going on during the compilation.

Though I did this on Linux, but it doesn’t really matter on the OS. The same syntax applies even to Windows.

With this you are done with compilation of your hadoop code. Jar your files and then execute them.

From my older blog: Quantum Computer-An Introduction

I wrote this article some 2 years back on my older blog. Now, since I made a shift of blog, I thought of including this particular article from my older one and since I am including this here after such a long span, I have added a few things and modified it a bit.

A guy once asked me that what is my limit to find a number’s nth root. I replied that I can find only the square roots and my speed is very slow. He then said, “Dude, you are not just slow, you are also very weak with numbers. Shakuntala Devi found the 24th root of a number so long that it might take you one whole minute at your fastest reading speed to read it, in mere 50 seconds and that too exactly! Shocked?!”

This is a simple example that shows that there exists many shorter and simple algorithms that we need to implement to make calculations easy.

Quantum computing is trying to implement all these algorithms. One of the newly found algorithms – Shor’s algorithm can increase calculations’ speed exponentially. By a few 1994 algorithm methods, it would take 8,00,000 years to factorize 250 digit number & 1025 years to factorize 1,000 digit numbers. Recently, a new algorithm was developed which can do this in few million steps.

A quantum computer is a device for computation that makes direct use of quantum mechanical phenomena, such as superposition & entanglement, to perform operations on data.

A classical computer has a memory made up of bits where each bit holds either a one or a zero. A quantum computer maintains a sequence of qubits (quantum binary digits). A single qubit can hold a one or a zero or importantly, a quantum superposition of these. In general, a quantum computer with n qubits can be in up to 2n different states simultaneously in contrast to single state occupied by normal computer. This is to say that in quantum phenomena, if there were 10 pens in front of me, and if I were asked which pen am I going to use for writing, the reply would be all 10 simultaneously. Interesting, isn’t it?

Simplest implementation of qubits can start with two spin states – “up” and “down”. Rest all states for a given qubit would be formed by a combination of these states only. But the fact is that any two discrete and sufficiently spaced consecutive Eigen values can be used for implementation of qubit like +1/2 and -1/2 or +1 and -1 or 0 and 1 on our number system.

For a two-bit register on a classical computer, the computer at any time is in any one of the four possible states – 00,01,10,11. Since, they are just discrete non-negative numbers (p,q,r,s), their probability adds up to give one.

But, when we consider a two-qubit register, we get a 4-D vector (p,q,r,s) called a wave function with complex coefficients. This complex coefficient brings the difference between the two vectors – one obtained from classical computation & the second vector of quantum computing. Now, since, this happens to be a wave function of complex coefficients, the sum of squares of coefficients’ magnitudes add up to give one. Being a wave function, superposition principle comes into play and we get interference pattern between different computational paths.

To say in simple words, in quantum computing we would have to keep track of all 2n complex coefficients for our system to manipulate data.

Quantum computation preserves the Euclidean norm, i.e., their sum of squares adds up to one. Generally, the quantum computational operations are rotations. Since, any rotation can be reversed, quantum computations are reversible.
After computation, a classical computer gives a definite two-bit string as a result. On the other hand, a quantum computer destroys the original quantum state. Quantum algorithm gives the result with a certain probability of accuracy. But, by repeated computation, we can get the most accurate answer (more or less like the one we do with the help of Numerical Analysis).

A quantum computer can very efficiently break many of the cryptographic systems in use today. They can also break through many of the so-called secure web-pages and emails. Thus, a quantum computer can be a very effective tool in breaking a large cryptic key and decrypt the code.
There also exists quantum cryptography that uses digital signature schemes to protect the data.

The attempt to guess the secret key makes a quantum computer a severe attacker on symmetric ciphers such as Triple DES & AES.

Not only this, quantum system problems (related to physics & chemistry) can also be solved by the help of quantum computing. It has been estimated that these computers can speed up the problem solving approach to such an extent that a year taking problem could be solved in seconds.

NMR techniques are the evolution of quantum computation. Molecular magnet and fullerene-based ESR computer can be future generation quantum computers. Some other future generation quantum computers can be Bose-Einstein condensate-based and spin-based quantum computers.

One thing that you would have marked in this article is that everything that I said that Quantum computer can do is written as “can” do. I mean I just suggested a possibility. This is because we directly work on atoms and the quantum entanglement which is not easy to control. And, you know the best part? The world record, until last year, for quantum computation was set by IBM and the problem was “3 * 5 = 15″! Yes, you read it right. The reason this was a landmark was that the computation was done totally on atoms utilizing its wave nature. Even a minor disturbance hampered the system.

With some advances, a supercomputer, JUGENE, did factorization on 42 bit quantum computers.

Going off the topic, a researcher recently developed a new form of chess that makes it difficult for even a supercomputer to estimate the opponent’s next move, thus helping a human to atleast let him compete.

Ending it, for the lovers of these two fields, quantum physics and computing, this definitely is a place to be and for the guys who want to do a research, this is the ideal platform to begin because the potential is huge and you have almost everything to discover and develop.

Using Windows Management Instrumentation(WMI)

Planning of scripting for Windows? The command shell, the Windows Scripting Host (WSH) and the Windows Management Instrumentation (WMI) provide a nice infrastructure for this.

The Windows command shell is better than ever (though I will say that UNIX still rules the chart when it comes to the shell that a UNIX administrator will gain access to). One can rapidly perform the task when the GUI capabilities aren’t required.

WSH is more an administrator tool, as per Microsoft. One can install almost any scripting engine that he wishes and get started with it. It just provides an environment for the scripts to run. Technically, it has no big and unique scripting feature. Though, I am not that big a scripting guy, I wont comment much on this.

The Windows Management Instrumentation almost gives you power to handle almost everything, right from your computer hardware to event management architecture to scriptable APIs. Yes, it’s a bit complicated to learn. A better alternative is wmic, a command-line interface, which comes with full documentation. With the short aliases, one can easily shorten lengthy WMI calls. A short simple example- let’s say that we have to see the number of processes running on my system and the amount of space they occupy. Type wmic on the command prompt and then type

process get name, workingsetsize

The output generated would be:

                          wmi

Now, one may ask why should we go for wmic when we can actually view this in Windows Task Manager and that too just with a few clicks, just no use of any freaking command? Well, now imagine you are sitting in a network and want to know what are the processes running on a node, say node1. You will just have to write /node:switch which is followed by the remote system’s name and then type this very same command.

/node:“node1”

process get name, workingsetsize

Want to delete a process? Type

process where name=<process_name> delete

or you can try the UNIX equivalent command

process [pid] delete

Well, the output format is tougher to read in this normal format. One can, therefore, view these outputs in both Excel or HTML format too. I personally prefer HTML among these two. To view the output in HTML format, one has to write

/output:E:\process.html

process get name, workingsetsize /format:htable

The output generated can be viewed by opening the web page ‘process.html’ which looks like

                                         wmi-html

For Excel, the format switch becomes csv.

Basically, everything that we do in wmic is decided by the verbs that are associated with its classes. The available verbs are

ASSOC Outputs Associates of the wmi object
CALL Executes a method
CREATE Creates a new instance and helps in setting its properties
DELETE Deletes the instance of the class
GET Retrieves the information about specific properties
LIST Lists the concerned data
SET Helps in modifying the properties of an instance

One can even manage the system configuration options with wmic. For an instance, let’s try

bootconfig get

The above command shows you the boot directory, configuration path and other related stuff.

As simple as that! Well, you can use WMI in GUI mode too. Just type wmimgmt.msc in command prompt and then go to the security tab in the properties of WMI control to handle the root directory.

Well I won’t talk all technical stuff because simply put, it’s almost impossible to talk about it on this very page. New shell options like Windows PowerShell are also available. Until I try it and write again, wmic can give you kick-start if you want to be a network administrator or try it simply because you love playing with your system.