After 2 days of struggling with Intel VTune on my Linux VMWare workstation and after trying several versions of VTune, I learned the hard way that you cannot get the Sampling data from VTune if you are running it on VMware.

I wish VTune would give me a better error message then just saying "Failed to create sampling data base. Probably .tb5 files are corrupted or don't exist." and "The Sampling Collector failed to collect data because the selected event(s) did not occur."

Looking at other people who were facing the same problem, the common suggestions were to try a different version of VTune. Yeah, thats where I wasted my time -- the error on .tb5 files was a bit misleading.

Thanks to Intel Forum post, I found and confirmed that Intel VTune is not supported on VMware. Here is another explanation from Intel on this. Having read these, it is understandable why VTune is not supported on any Virtual system. With Virtulization technology becoming so popular among developers, I wish Intel can do a bit more to let know that it is not supported.

The only good news is that you can still generate call graphs using VTune on VMware, but thats not the reason why you want to use VTune.

Today discussion is around programming in C/C++ and file IO.

Almost all programs require reading files from disk to perform their duties. I have been using fread(), read() or C++'s fstream classes to perform the file IO. The way these file IO works is : Data is copied from the disk to a kernel buffer, then the kernel buffer is copied into the process's heap space for use. The data as you see is copied twice. These file IO is always buffered and the amount of buffering differs from one OS's implementation to another.

This works good for small files, files which programs read for configuration details and writing log files, etc.

But what about large data file? A file which is in the order of a few MBs can choke the system. In most cases (maybe except for photo editors, or video programs) it is generally not required to get the entire data for a particular operation. We sometimes need to access random parts of a large file at random times.

Try to solve this problem with the tradition fread/fseek calls and you will end up wasting too much CPU and memory.

The concept of memory mapping of file allows the program to copy the data from the disk straight to the process space. This allows the program to view the contents of the file as an array -- you get the pointer to the first byte of the file. This translates to better usage of CPU and RAM.

In UNIX these can be accomplished by the mmap() function and in Windows its the MapViewOfFile() function. Using these functions is not as easy as using the traditional reads as the programmer may need to take care of certain things:
1. You may need to map only a particular section of the huge file into memory and need to do the alignment with system pagesize.
2. Delicate pointer handling and proper usage of read write permissions.
3. Make sure you dont map a very huge file into your process space, which can lead to too many page faults.
4. Never write pointers into these files. Keep only offsets if you need.

It also allows you to dump serialized objects which you can directly load into memory and cast it to the required type and start working right away.

This technique is only suitable for certain cases and the programmer must make sure that he/she gets measurable advantage using memory mapped files.

Suggested Reading:
1.Sun Developer Network : A Performance Comparison of "read" and "mmap"
2.Pros and Cons of Memory mapped file and when not to use memory mapping.
3. Memory Mapped Files And Shared Memory For C++.

All Unix users are aware of the DAC (-rwxrwxrwx) that we always use to prevent others from accessing files we don't want them to. But there are times when you want more than that.

You may say:
1. I want to prevent myself from accidentally deleting a file despite having -rwxrwxrwx permissions.

2. If you have 000 permissions for a file, then you cant delete it without changing the permissions first. But you can straight away do a "rm -f" .Many people are just used to using the "f" flag(I sometimes wonder if the "f" flag meant "force" or something else). If you are a Linux sys-admin and you are sharing the root password with fellow admins, then you definitely need something to warn your fellow-admins from doing something wrong accidentally. The traditional DAC does not allow you to this. For DAC "root" is as good as God.

3. I want to prevent every body (including root) from deleting the file or writing to file, BUT YOU WANT TO ALLOW THEM TO ADD DATA TO FILE. ie, allow only appending.

A very less know Linux command "chattr" allows you to do precisely these by making the file immutable. This works only for EXT2/ETX3 file-sytems only. If you set the immutable attribute for a file, it means that it cannot be modified, deleted, renamed and no hard links can be created to this file.

Even a root user attempting to delete this file with get the message "Operation not permitted". Only the superuser or a process possessing the CAP_LINUX_IMMUTABLE capability can set or clear this attribute.

chattr +i <filename> sets the immutable attribute of the file.
chattr -i <filename> clears it.
lasttr <filename> will show the status of the attributes.

This not-so-popular command has got a bunch of other attributes that you can set either to improve the security of your system or even to increase performance.

Using the +a attribute you can make the file to be opened only in append mode. (-a to remove the attribute.

For a security sensitive file-system you may want to set the "+s" attribute which will make sure that when you delete the file, the contents in the hard disk are all zeroed out.

Read the man pages of lsattr and chattr for more details and limitations/bugs. The full features of chattr are not yet available.

As promised in my post on Thoughts on password management here is a step by step guide on how someone can capture your login id and password while you login into a website that does not support HTTPS (HTTP over SSL).

A little basics first. HTTP protocol is based on clear text interchange of data. It is an application layer protocol with TCP and IP in the underlying layer. It means that anybody who can capture a HTTP packet traveling over an Ethernet network can read all information you sent.

When you type http://blog.netotto.com/index.php, your browser sends the following request:

    GET / HTTP/1.1\r\n
        Request Method: GET
        Request URI: /index.php
        Request Version: HTTP/1.1
    Accept: */*\r\n
    Accept-Language: en-us\r\n
    Accept-Encoding: gzip, deflate\r\n
    User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)\r\n
    Host: blog.netotto.com\r\n
    Connection: Keep-Alive\r\n

This is a HTTP GET request, the important part is just the GET and the Host lines. The server responds to this again in clear text, that your browser can understand. Here is how it looks:

   HTTP/1.1 200 OK\r\n
        Request Version: HTTP/1.1
        Response Code: 200
    Date: Sat, 15 Sep 2007 10:41:22 GMT\r\n
    Server: Apache/2.0.54 (Unix) PHP/4.4.7 mod_ssl/2.0.54 OpenSSL/0.9.7e mod_fastcgi/2.4.2 DAV/2 SVN/1.4.2\r\n
    X-Powered-By: PHP/5.2.3\r\n
    Expires: Thu, 19 Nov 1981 08:52:00 GMT\r\n
    Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0\r\n
    Pragma: no-cache\r\n
    Vary: Accept-Encoding\r\n
    Content-Encoding: gzip\r\n
    Content-Length: 6611\r\n
    Keep-Alive: timeout=2, max=100\r\n
    Connection: Keep-Alive\r\n
    Content-Type: text/html\r\n
    \r\n

    Content-encoded entity body (gzip): 6611 bytes -> 26607 bytes

This is followed by the HTML page (index.html) in clear text.

If you are at the login page of a non-secure web page your login and passwords can easily be read by somebody on the network. All you need is a network packet capturing tool.

On an Ethernet, the electrical signals (to which your HTTP packet finally resolves to while traveling on the wire) is actually readable by all the computers in your Ethernet LAN. The network card in normal operation mode (called non-promiscuous mode) does not read packets that are destined for somebody else.

A packet capturing tool like Ethereal can ask your network card to be in promiscuous mode and read all packets. These includes packets destined for other machines in your LAN.

You can download and install Ethereal for Windows from the Ethereal Download Page and install it and follow the following steps to convince yourself that sending data over a non-secure channel (a non HTTPS website for example) is very un-safe.

Lets jump straight into the steps:

In this example, I have installed Ethereal on the same computer from where I will be attempting to login to a non-ssl website like www.youtube.com . You can however run the Browser and Ethereal on different computers in your LAN which are in the same subnet (Eg: Browser on your computer at office and Ethereal on your colleague's computer).

1. Open the youtube login page (http://youtube.com/login?next=/index) in your browser. Note: its http and not https. I type in "shuva" and password as "testpassword"

2. Start Ethereal (I am using version 0.99.0)

3. Lets configure it so that it captures only HTTP packets. We dont want to capture hundreds of other packets. From the menu bar, select Capture--->Options. In the Options dialog box, choose the correct n/w interface if you have more than one. I have something like "Intel Pro/100 VE". In the capture filter type in "tcp port http". This means we will capture only http packets.

4. Click the start button to start the capture.

5. Click on the login button on your browser.

6. Back to Ethreal, click the stop button on the popped up dialog box.

The Ethereal window is divided into 3 parts,

the upper part listing the packets,

the middle part showing the selected packet in human readable form and properly divided into the layers (HTTP/TCP/IP/Ethernet/Frame) and

the lower part showing the raw bytes of the selected packet.

In the upper part, where you many lines with lots of IP addresses, search for the line that has "HTTP" and "POST /login?"

In the middle part click on "+" sign that says "Line-based text data". There is your login and the password. Just above the "Line-based text data" is the HTTP request that you sent to youtube.

Here is a screen shot:



Below is the HTTP request that went through the network:

    POST /login?next=/index HTTP/1.1\r\n
        Request Method: POST
        Request URI: /login?next=/index
        Request Version: HTTP/1.1
    Host: youtube.com\r\n
    User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6\r\n
    Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5\r\n
    Accept-Language: en-us,en;q=0.5\r\n
    Accept-Encoding: gzip,deflate\r\n
    Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\n
    Keep-Alive: 300\r\n
    Connection: keep-alive\r\n
    Referer: http://youtube.com/login?next=/index\r\n
    Cookie: dkv=; GEO=a78121630c1e9382b3cf8d384cd0d399cxYAAABJTixhcCxoeWRlcmFiYWQsLCwsLC0x; LOCALE_PREFERENCE=86d1d09eefe6b79b4068000ce05518a4dAUAAABlbl9VUw==; LOGIN_INFO=; VISITOR_INFO1_LIVE=D6Jub46OfWo; use_hitbox=72c46ff6cbcdb7c5585c36411b6
    Content-Type: application/x-www-form-urlencoded\r\n
    Content-Length: 93\r\n
    \r\n
current_form=loginForm&next=%2Findex&username=shuva&password=testpassword&action_login=Log+In

Disclaimer: My selection of youtube.com for this demo is not to prove that youtube.com is unsafe. I just wanted a very well known website. My intention of writing this article is not to encourage people to read somebody's else packet from the network but to demonstrate to users that visits to non-secure sites are not safe and you should therefore never select a password for such sites which you are reusing for a secure site like your email or banking sites.

Relevant posts:
1. Thoughts on password management.
2. Storing all your password safely.

I have been searching for a good free XML editor which this two basic requirements:

1. Allow editing XML files in code and also allow me to expand and collapse sections (like IE does)

2. Can perform XSD validation.

Having used Altova XMLSpy in my last job, I knew something thats free cannot probably have all features like XMLSpy.

Unfortunately I landed up no-where.

I started with Microsoft XML Notepad, but it wont allow me to edit in code. Otherwise it does have a lot of features. Its very important for me to edit in code.

Then I installed Cooktop but after using it for a few minutes I realized that when a XSD validation fails, the error messages that gets popped says that it could not find the definition of the root element in the XSD file. It wouldn't pin-point the error.

It means if you type in a string for a attribute which is defined as xs:integer, validation thinks that the root element is not defined. Na! Not good.

Then I tried using with my favorite editor, Xemacs but just could not figure out how to make it validate against .xsd. I found a lot of help to validate against DTD and SGML validation, but not XSD.

However in this process, I found a very useful comparison chart of all the XML editors available. Its very exhaustive and the comparison is too detailed. Here it is:

Choosing an XML editor by hijs van den Broek.