Virtualbox crashes on host memory too low – this time it IS the third party software to blame

Oracle Virtualbox is a great and convenient VM hosting application under GPLv2, and I use it to run Linux on my Windows laptop.  However, a while back, I couldn’t get it to run for more than 5 minutes — for no apparent reason at all, it would pop out an error message saying host memory too low, and the guest Linux would just freeze, and the only thing to do would be powering off the VM.  But as soon as you boot it up, it would happen again.

Several other people encountered the same problem according to the virtualbox bug tracker.  A similar bug report was filed a few years back and the problem was fixed, besides it was a few versions back and people were running Virtualbox on typical 32-bit Windows XP machines with 2GB of memory.  But things are quite different now since most computers are 64-bit equipped with 6~8GB of memory, and there is no reason the host memory should run low (if it could work on a 32-bit host with 2GB of memory, why would it not work on a 64-bit host with 6GB of memory when the guest OS is 32-bit?).

A couple of folks suggested that changing the BIOS settings to enable hardware virtualization would solve the problem, but it has already been set on mine.  Finally, one comment pinpointed the cause – it was Google Crash Handler.  During the course of updating things, I must have enabled “reporting anonymous usage stats” in Chrome.  As soon as I disabled it, VirtualBox start to function again.

So, just as I said earlier, let’s keep blaming the 3rd party software!

The mystery of Windows Experience Index fail to assess disk performance

I have posted this before on another blog space, but decide to repost it here as I intend to use this blog exclusively from now on.

So I bought a new Dell XPS laptop with core i7 running 64-bit Windows 7 almost a year ago.  After getting everything up and running, I couldn’t get Windows Experience Index (WEI) to compute the score and it kept failing at the very last step which was assessing the disk performance, saying something about data being invalid.  Granted this number doesn’t mean much, but still I felt as if something was missing.

To add to the confusion, the disk went bad within just a week (says a lot about the quality of XPS, which is supposed to be a high-end product line).  So at first, it seemed to make sense and I thought no wonder, because the disk was bad.  Then Dell sent out a technician and replaced the disk with a refurbished one (just like buying a new car, as soon as the machine left their warehouse, it started to lose its value and would only deserve refurbished replacement parts), but WEI still would not work.

Searching around a bit online and found a couple of other folks encountered the same problem, but there was no clear explanations nor solutions.  Finally I decided to bite the bullet and sent a help request to Microsoft’s customer support team.  The guy who got back to me immediately started to blame third-party software.  There are three words exactly describing how I felt when I saw his response: predictable, ignorant, and funny.  I guess it was their SOP to immediately pointing the finger at some unspecified “3rd party”, but what he didn’t know was that every time I get a new computer, I wipe out what came with it and do a clean installation using my own OS (a sincere thank to the deal between my university and Microsoft, we get Windows Ultimate edition at an extremely low price), so it was a clean installation of Windows 7 without any other software on it.  Even if it were true, that is, some 3rd party software indeed caused the problem, it didn’t make WEI look any better because it just show how vulnerable WEI can be.

As everything else seemed to be working fine, I decided to let it be since this WEI number really doesn’t mean anything.  Then just a few month ago, I saw another post complaining about this problem, but this time, the gentleman also found the cause.

Just like what I typically do, this gentleman re-installed his own Windows 7, but more importantly, he also moved all User directories to a different partition than C:\, and created symlinks on C:\ to point to the actual directories.  As it turns out, while the symlinks work fine for other applications, it does not work for WEI.  Instead, WEI relies on two environment variables TMP, and TEMP, to dump its I/O data, and by default, both TMP and TEMP point to the temporary directory within the AppData directory under each user.  Since the user directories are relocated to a different partition, and WEI doesn’t honor symlinks, it is unable to find the physical directory pointed to by the env vars and thus the error.  As soon as I reconfigured these two env vars to point to the physical temporary directories instead of through the symlink, WEI started to work.

So yes, keep blaming the 3rd party software.

How to limit certain ports to specific source IPs while keeping all other ports open using iptables

Most online tutorials on iptables suggest to block all ports and DROP all unmatched traffic by default.  While such configuration is most secure and should be done on many home users’ and production hosts, it does not work in our environment.  Many of the machines in our lab are development machines, which means any of us can throw some services or containers on the machine, pick an arbitrary port to host them and access them from somewhere else.  But every once in a while, we feel that the access to certain services should be limited to a few specific source IPs, so we want to block these ports to all traffic except those from trusted nodes, while keeping other ports open so other folks can continue do their work.

Our machines run Redhat Enterprise Linux, and the default iptables rules found in /etc/sysconfig/iptables look like the following:

*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:RH-Firewall-1-INPUT - [0:0]
-A INPUT -j RH-Firewall-1-INPUT
-A FORWARD -j RH-Firewall-1-INPUT
-A RH-Firewall-1-INPUT -i lo -j ACCEPT
-A RH-Firewall-1-INPUT -i eth0 -j ACCEPT
...
-A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited
COMMIT

The key here is the line just before the “…”, which says the firewall shall ACCEPT all traffic coming from the network interface eth0, and this is what makes our dev machines quite open.

The line after the “…” and just before “COMMIT” is a secure catch-all that REJECTs all traffic that does not match any rules defined above.  In the case of this particular machine, this line has no effects because the machine has only one network card.  However, if our machine had other network cards (e.g. eth1), then all unmatched traffic coming from these interfaces will be REJECTed.

The difference between DROP and REJECT is that DROP just ignores the request, whereas REJECT sends a message back to the requester.  It is preferably to use DROP to not create extra traffic load to your network.

Now suppose we have a service that listens on TCP ports 65000 and 65443, and we want to limit its access to IPs 100.101.102.103, and 200.201.202.x, where x can be 96~111.  The first thing we want to do is add some extra blank lines before the “ACCEPT all traffic from eth0” line, because all our rules must go before this line into the blank area.

The first two rules are to ACCEPT traffic from these two IP ranges:

...
# RULE 1:  accept requests from my best pal Joe's workstation, besides, he paid me $50 to have access to this service
-A RH-Firewall-1-INPUT -m comment --comment "Joe volunteered to test this service"  -s 100.101.102.103 -p tcp -m multiport --dports 65000,65443 -j ACCEPT

# RULE 2: accept requests from other workstations from our lab.
-A RH-Firewall-1-INPUT -m comment --comment "this service is available to all lab workstations" -s 200.201.202.96/28 -p tcp -m multiport --dports 65000,65443 -j ACCEPT
...

A few things worth noting here. There are two places to add comments. One is the use the typical Linux hash in the rules file itself – this comment is visible only by looking at the file. The other place is using the -m comment extension. This comment will show up if someone runs the command /sbin/iptables -L.

Here we also use the -m multiport extension to combine all ports used by this service in the same rule. Without it, you would have to list each port on its own line using –dport (notice the singular form).

Since RULE 1 involves a specific IP, it is listed after the “-s”.  But in RULE 2 we have a number of IPs.  It happens so the range of the IPs go from 96~111, which in their binary form are 01100000 ~ 01101111, so we can conveniently use a subnet mask of 28 bits to mask out the last 4 bits.   However, if the range of IPs does not fall nicely into a subnet range (which is probably the more common case), you could use the -m iprange exension instead:

# RULE 2 alternative using -m iprange:
-A RH-Firewall-1-INPUT -m comment --comment "this service is available to all lab workstations, including the workstation for our new intern Asok 200.201.202.112" -m --src-range 200.201.202.96-200.201.202.112 -p tcp -m multiport --dports 65000,65443 -j ACCEPT

Now we need to add a rule to DROP all other traffic to these ports. If we don’t add it, the “accepting all traffic on eth0” rule below would still make our service wide open.

# RULE 3: drop all other requests to the service
-A RH-Firewall-1-INPUT -m comment --comment "no one else allowed to use this service" -p tcp -m multiport --dports 65000,65443 -j DROP

Save this file and restart iptables:

> sudo service iptables restart

Below is the complete listing of /etc/sysconfig/iptables

*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:RH-Firewall-1-INPUT - [0:0]
-A INPUT -j RH-Firewall-1-INPUT
-A FORWARD -j RH-Firewall-1-INPUT
-A RH-Firewall-1-INPUT -i lo -j ACCEPT
...
# RULE 1:  accept requests from my best pal Joe's workstation, besides, he paid me $50 to have access to this service
-A RH-Firewall-1-INPUT -m comment --comment "Joe volunteered to test this service"  -s 100.101.102.103 -p tcp -m multiport --dports 65000,65443 -j ACCEPT
# RULE 2 alternative using -m iprange:
-A RH-Firewall-1-INPUT -m comment --comment "this service is available to all lab workstations, including the workstation for our new intern Asok 200.201.202.112" -m --src-range 200.201.202.96-200.201.202.112 -p tcp -m multiport --dports 65000,65443 -j ACCEPT
# RULE 3: drop all other requests to the service
-A RH-Firewall-1-INPUT -m comment --comment "no one else allowed to use this service" -p tcp -m multiport --dports 65000,65443 -j DROP
...
-A RH-Firewall-1-INPUT -i eth0 -j ACCEPT
-A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited
COMMIT

Reviving this Blog

It dawned on me today that I have this blog space after Microsoft decided to close out their “live spaces” and transferred everything here.  It would be a shame to let it go wasted.  So I have decided to revive this blog and start posting things, meaningful things, on topics such as software development and photography.

How to create a TF2 spray with transparency for FREE!

There are a few tutorials online on how to create a Team Fortress 2 spray decal with transparency, but as far as I know they all have problems.  Some of which may have worked (for example, [this one]), but they start it off by telling you to create the image in Photoshop CS3, and then use a free software called [VTFEdit] to make it a decal.  Not every one can afford to buy Photoshop.
Another [tutorial] is supposed to show you how to create an animated decal, but the author seems to lack the fundamental knowledge regarding image formats, and tries to convince people that JPEG supports transparency. (Hard to believe this tutorial was endorsed by MaximumPC. Major doodoo on their part.) Besides that, the author uses Photoshop as well.
But, there is a totally FREE way of creating a spray decal with transparency.  Forget about Photoshop, or even the freeware VTFEdit (well, unless you want to create animated decals, but this is not what my blog entry is about).  All you need is a free graphics software called [Paint.net], and the key is to save your image as a 32-bit uncompressed TGA.  Now if you know what I am talking about, you are free to go and create your own decals.
For the rest of you readers, I will be showing the steps in making such a decal.
STEP 1.  Of course you need to download and install paint.net first.
STEP 2. Create a new image, with width and height both being multiples of 16.  For example, 64×256, 128×80 etc.  Make sure to disable the anti-aliasing so you can construct your design using scratch lines and remove them easily by recoloring in the end.
STEP 3. Add a new layer.
STEP 4. Delete the background layer, so that only the newly added transparent layer is present. (You may choose to delete the background at the very end, but I personally prefer getting rid of it up front)
STEP 5. Do your magic – be artistic and creative, and make your decal as wierd as possible.  Here is what I created.  Notice the gray-and-white checker patterns that indicate transparency. 
STEP 6.  Save your image as a 32-bit uncompressed TGA.  Make sure to uncheck “Compress (RLE)” or TF2 won’t be able to recognize the file format.  In reality, you may want to do this step as often as possible throughout your creative process (in other words, save you files often so you won’t cry for mommy later when a sudden power failure takes place).
STEP 7. Fire up TF2.  Go to Options, under the tab Multiplayer, you will be able to import the decal.
STEP 8. Test it out in a TF2 map.


why TF2 has the upper hand over FF

Team Fortress 2 definitely has the upper hand over Fortress Forever.  You can tell by simply counting the number of servers running for each game: 3000 vs 40.
 
Coming from TFC, and before had my hands on TF2, I was a bit dubious about the changes Valve made to TF2, especially the lack of grenades in the game.  Spamming with grenades is officially an annoying "newbie" thing to do in TFC, but we have all done that because it is FUN!
 
TF2 also changed the artistic style from a realistic environment to a "cartoonish" environment and caused more doubts to be casted over its success. (For crying out loud, people even complain when the gamma level is set too high in the upcoming Diablo III game).
 
But all my doubts disappeared as soon as I started playing TF2.  There is really no need for grenades – just like the developers at Valve explained, without grenades, each player tends to focus more on what they are supposed to be doing with their specialized skills.  The "cartoonish" style makes each class more lively, and gives each one a unique personality – the muffed calls from pyros, the weird affection between the heavy and his gun, the somewhat vulgar but definitely hallarious taunt from the demo – all added special touches to the already addictive game play. 
 
Fortress Forever, on the other hand, lacks all this.  My first impression of FF was that "wow this game is soooo frigging quiet!"  You can run for miles in FF without hearing anything.  zzzzz… … … The character animation definitely needs more work – the movement of the models looks too robotic.  The worst and most annoying part is the mechanical voice of the announcer.  BLUE. TEAM. CAPTURED. FLAG.  YOUR. FLAG. IS. STOLEN.  Automated customer service phone systems don’t even do this anymore, why would FF go this route?
 
FF boasts a "realistic" environment as one of its selling point, but I find everything rather "artificial", far from being "realistic".  TF2 on the other hand, looks more realistic with barnyards and train yards.
 
Therefore, TF2 is definitely superior as of now.
 

a few notes on vista

finally got the machine to the way I wanted.  just want to jog down a few quick notes on vista ultimate.
 
Pluses
  Open command window here is now built in.  The secret is to right-click while holding down the "shift" key, so it will show up in the context menu.
 
  Symbolic links. this is probably something a linux user has long been waiting for.  although it is said symbolic links had been present in NTFS for quite a while, there hadn’t been any user commands for making them.  But now, you can always use mklink to create a symbolic link (or even a hard link, if you desire).  This allows me to alleviate some burdens off from the C partition – software such as Windows Live Mail doesn’t allow a user to choose where to install the program or where to save the mail files.  By using symlinks, I can create a directory on a bigger partition (D:\ for example), and then symlink it from C:\
 
  However, it is tricky sometimes because a dos command doesn’t have administrative previliges, so the trick is to create a shortcut to cmd.exe, open the window under "RunAs administrators", and then the shell will have all the administrative powers you need to symlink.
 
  procedures for symlinking Windows Live Mail to a different partition
  – Create a folder on D:\ (or any other partition that is big enough), let’s name it  Johns_Email
  – Copy everything from  C:\Users\John\AppData\Local\Microsoft\Windows Live Mail\  to D:\Johns_Email
  – Rename Windows Live Mail folder on C:\ partition to something else, such as __Windows_Live_Mail
  – Open cmd.exe using administrative previliges
  – cd c:\Users\John\AppData\Local\Microsoft
  – mklink /d "Windows Live Mail" "D:\Johns_Email"
  – Run Windows Live Mail application and make sure it works
  – Delete C:\Users\John\AppData\Local\Microsoft\__Windows_Live_Mail
 
Minuses
  Many applications do not work, including Nero Burning Rom prior to version 8.  And it is a good opportunity for money-hunger companies to abandon earlier versions of their app, and force consumers to buy their latest "vista-compatible" version.  Shame on them.
 
   The old tweakUI and powertoys are also among the missing ones.
 
  There seems to be a glitch (or limitation?) with copying.  It seems the filename + path cannot exceed 260 characters.  To get around this, use a compression application such as WinRAR, to compress the data you want to copy, and then uncompress it to the destination.  Do not use Windows’ built-in zipping tool, because it suffers the same fate as copying.
 
   It is annoying that virtually everything you do will trigger the permission dialog box to pop up.
 
   Aero seems to be focusing too much on its "wow" effects, but missed the fundamental functionalities.  No matter how good your desktop looks, the screen size is still limited, and what users want is a simple Virtual Desktop to extend the screen property.
 
   File Sharing using the wizard is confusing (for me, at least), and I still went back to the old XP/2000 way (less secure??)
 
   Another thing keeps me wondering is the login screen.  There seems to be no way to remove the welcome screen for logging in. While it is okay for a home user because the list of user accounts will be small, how can it be used for a large institute with thousands of users?
 
Neutral
 
   Bye FDisk.  Hello DiskPart. (seems to be present in XP as well, but honestly I didn’t know that)