IE favorites, NTFS symbolic links and Dropbox
NTFS symlinks
Well-known to *nix-users, unknown to most Windows-users.
A file or folder can exist in more than one place, meaning the file is stored once to disk, but can have two different names on different locations. If you change it one place the change happens to the other place. I won’t bother going into details, but in short NTFS supports a few different ways to do this.
New in Windows Vista and Windows 7 is “NTFS Symlinks” which is a powerful way of linking files and directories between filesystems and over network in addition to the usual duality.
Dropbox
Dropbox is a great tool for synchronizing small amounts of files between computers (<2GB free).
Symbolic Link Creator
This application gives you a simple to use GUI for creating symlinks.
(Quick hint: Run as Administrator)
Synergy effect
Dropbox does a great job in synchronizing the dropbox folder, but it requires you to actively put files there. This is fine for all the stuff you can choose, but there are a few scenarious where you can’t do that easily.
In the picture above I have created a “_MachineSync” folder that contains a Favorites folder. By deleting my Favorites-folder in my user profile and linking in the Dropbox Favorite-folder all my IE favorites is synchronized to Dropbox. I’ve done the same on both my work PC and home PC, so now favorite items are instantly synced between home and work PC (as well as being accessible via web).
The same could be done with my documents, Desktop or anything else I’d like to keep in Dropbox. Just note that some folders require you to log on as a different user to be able to delete them before symlinking them in from Dropbox.
Optimizing RDP for casual use (Windows 7 SP1, RemoteFX)
Using RDP
Since some time back around 2000 I’ve been running a dual desktop configuration at work. One screen for my laptop at work, and one screen with RDP home to a more powerful computer. Since I spend most of my time in front of a computer I usually invest a fair amount of money when buying a new computer, so my home computer is often the powerful one.
The problem
The problem is (for the most part) bandwidth. If it was over LAN then RDP would have no problem pushing a video, but over slower links you can notice each screen update. Currently I have a 26/4Mb line (waiting for a 100/100Mb) at home, so the screen of my home computer is being sent to me on a 4Mb link. That should be enough for casual use, but not full screen video. (The reason being that RDP is sending a lossless image and can’t real-time mpeg encode the picture because of CPU-power, latency in encoding and lossy encoding – so it has to use lossless compression that is easy on the CPU.)
A secondary problem is GPU hardware. Today most applications use hardware acceleration, but when you work via RDP a virtual (purely software) graphics driver is used to render your content. This worked for pre-Vista desktops, but it’s now few years since Windows switched from the obsolete memory rendering technique to a new hardware driven rendering and applications are starting to adapt. In short hardware accelerated rendering allows applications to render their content faster using less CPU.
A third and very noticeable and annoying problem is that RDP tries to send all the frames. If visit a web page with animated banners, list a whole lot of stuff in Putty or other things that update the screen a lot you have probably noticed that the RDP session hangs and that the best way to get it back is to minimize the window (because then it doesn’t send any data and the screen can finish updating).
The solution
RDP can be configured through group policy. The first and most straight forward setting to note is compression level. Default set to save memory. Setting this to save bandwidth will … hold on … save bandwidth!
Second is a new feature in Windows 7 SP1 and Windows 2008 R2 SP1 called RemoteFX. This even works on virtualized guest os running on HyperV. In short it allows RDP sessions to use hardware acceleration for rendering. It also changes the sampling method from “every update to the screen” to “interval update”. So if your screen updates a lot you will get only every x frame (as opposed to earlier where you had to wait until all the frames had been transferred). RemoteFX will use GPU for compressing the image being sent, making your computer (CPU) faster as well.
The difference explained in short: Going from seeing screen updates happen and window freezing to everything being smooth, responsive and even being able to (almost) watch a YouTube video smoothly.
Procedure
Note: This required Windows 7 SP1 or Windows 2008 R2 SP1. But some settings can be tweaked for earlier versions, so it is still worth following the instructions as far as possible.
- Start: gpedit.msc
- Go to: Administrative Templates > Windows Components > Remote Desktop Services > Remote Desktop Session Host > Remote Session Environment
- Configure RemoteFX: Enabled
This will enable GPU processing if your hardware supports it. GPU processing will speed up modern browsers, most apps, WPF apps, video, etc. - Optimize visual experience when using RemoteFX: Enable and set both settings to “Medium” (or whatever suits you best). Feel free to experiment.
As opposed to earlier (default) settings where RDP tries to send every frame this allows RDP to skip frames to keep up and not freeze. - Set compression algorithm for RDP data: Enabled and set algorithm to “Optimized to use less network bandwith”
- Optimize visual experience for Remote Desktop Services sessions: Enabled and set Visual experience to “Rich Multimedia.”
- Reboot and feel tha powah!
Small note: When I was first researching RemoteFX in SP1 beta I read that you had to enable RemoteFX on the client as well. I am uncertain if this is true as I do it anyway (since it also it RDP host sometimes), but I highly doubt it. Enabling it on the RDP host should be enough. However setting it on both only takes a few seconds, so …
Bonus
If security is not a big concern you can disable encryption to speed things up. I haven’t tried this, but from what I read while researching RDP speed issues it should speed things up. Setting “Set client connection encryption level” under Security to “Low Level” should enable encryption for sending keys/mouse and disable encryption for receiving image.
Screenshots
Should I defrag my SSD?
The answer is: “Yes!” and “No, never!” or somewhere between.
SSD disks has overcome the physical limitations of traditional harddisks. It is actually not a harddisk any more because it doesn’t have a disk. Traditionally when you access a fragmented file the data is scattered around on the disk so the read heads has to move around on a magnetic surface to read the whole file. With SSD this is no longer a problem as the data is stored in NVRAM (think “USB Pen”). This is one of the most significant advantages of the SSD drives. So fragmented files can be read nearly just a fast as defragmented files.
However the NTFS file system does some interesting things.
All hail NTFS, it was a very good file system at a very early time. Others have only started to catch up recently. Though there are some really good alternatives, none (afaik) has all the capabilities that NTFS has.
NTFS keeps track of all allocated and free space on the partition. When storing a file a certain amount of blocks are allocated. If the file size is only one block size it is actually just stored directly in the MFT. But if the file requires multiple blocks then free blocks are located and used. NTFS tries to find areas with many free blocks to prevent fragmentation. Despite this conditions such as high fragmentation (only small sequences of blocks available/low on space) and simultaneous writing to the disk may lead to fragmentation. Most of the time Windows don’t know how big the file you will be writing is and therefore can’t reserve the space. Other files can then be written in between causing fragmentation.
So when the time it takes to read a fragmented file isn’t an issue, how can fragmentation be an issue? One great feature in NTFS is that it will store allocated and free data as chunks. Traditional file systems like for instance FAT32 has a 1-1 mapping of file allocation and data block. NTFS does this way smarter by storing sequences. Instead of storing “FILE 100 is allocated to blocks 10,11,12,31,32,33” it stores “FILE 100 is allocated to blocks 10-12,31-33”. For a defragmented file it only needs one allocation sequence, while for a fragmented file it will need many.
The end result is that a defragmented drive will keep the MFT (NTFS file table) smaller so less memory is consumed, more of the (frequently used MFT) can be cached in memory and less CPU is required for processing a file.
Now the problem is that SSD drives has a limited number of writes for each block. Defragmenting the disk regularly will use all the available writes pretty quickly.
My recommendation is to defragment the SSD disk very rarely. You want to keep the files you never write to defragmented, and keep free sequential space to write files in. Defragmentation programs that can leave defragmented files and only worry about the fragmented ones is a good alternative. Defrag that comes with Windows will do fine.
Switching from HDD to SSD
A couple of days ago I bought a 60GB SSD disk for my primary computer (home desktop). It made a world of difference. I thought I’d share some info on how disks works and what the differences are. The aim is to keep this post at a level where people with a medium understanding of computers and hardware can keep up.
My current system
My computer is a custom built quad ~3GHz, 8GB thing with a Asus Republic of Gamers Maximus Formula motherboard. Fast and shiny a few years back, still kicks ass. I have two graphics cards: ATI 5700 something (800 stream processors, 1 or 1,5GB RAM?) and an older NVidia 9600GTO (96 stream processors, 768 MB RAM). Most games run at top resolution without any problem. 4 harddisks with a total of 4TB space. Two 1,5TB disks are in mirror, these were also my system disks.
I’m a poweruser. By that I mean I install a lot of programs and SDK’s. I even use Interix. So I have 1,496,388 files on my computer – much of which is Cygwin/Interix/ SDK’s/backup of old installations, etc.
I must admit I’m fairly interested in hardware. I know much about how stuff works in the really low levels like signaling, frequencies, CPU and motherboard architecture, communication between components, operating system drivers, etc. I’m a developer that consider L1/L2 caching when writing code and see the code in number of instructions rather than functionality.
I figured that with 8GB of RAM I keep most of the important stuff in cached in memory, so the difference with SSD after the initial boot would not be that great. I have been planning to upgrade to 12GB or 16GB, and I’m looking into buying a whole new system. But that is usually a big investment of around 12000 NOK (just over $2000) for just the computer (chassis, power, MB, CPU, RAM, HD, graphics).
Some thoughts on HDD
Old-school harddisk has some problems.
- If you compare the HDD with everything else in your computer then the size of the disk has increased over the past 20 years, but compared to other components the speed has been standing almost still.
- The evolution of PATA, SATA, SATA2 and SATA3 allows the computer to communicate faster with the disks. But the disks are not actually that much faster, they use ‘cheats’ to make them appear faster.
- Write cache and read-ahead is one such cheat that help speed up things. Write cache allows Windows to make a relatively small write to the harddisk without waiting for the data to be physically stored. Read-ahead is a feature where the harddisk reads more data than was requested because it was there anyway so the extra read-time required for that is so small compared to the statistical probability that the file is in sequence (not fragmented) and that the app will request more data from it. Remember that the harddisk has no idea of what a filesystem is, so it will not skip read-ahead for small files or understand fragmentation.
- Windows also has write caching which uses RAM to queue disk writes. The main purpose of this is to avoid many small writes, say for example if two or more applications are writing at the same time. By waiting a little bit many small write operations can be combined into fewer big write operations.
- Windows supports extended write-behind caching so that it can help the harddisk write out data in sequence using the elevator algorithm to avoid moving the disk head too much, but this is dangerous as it both breaks NTFS journaling and can result in a massive loss of data if power is cut or Windows crashes.
- Servers usually have onboard RAM on the RAID-controller with battery backup in case of power loss. This allows it to continue writing the data when power comes back.
Enter SSD
So I bought a 60GB SSD disk. My expectations were:
- This should speed up things. From around 30MB/s actual speed to somewhere over 285MB/s read and 275MB/s sustained write (at least that’s what the spec said).
- Seek time is gone. This is the time it takes harddisk heads to move around the harddisk.
- My old drives will be worn out, and even if not used it will be demagnetize and loose data in a few years. My SSD will “last forever” (or at least 20 years+).
- SSD’s do get worn out by write-operations, but that is handled automatically so there is no loss of data.
What I discovered after installing it (subjective observation):
- Installing Windows 7 Ultimate from USB pen to SSD disc took 8 minutes. Really! 8 minutes until the desktop was ready for use.
- Windows boots in around 20 seconds. When the desktop comes up it is immediately ready for use. No slow loading of apps in the background.
- Suddenly my BIOS feels slow. Before I didn’t notice it because Windows was so slow to boot.
- Quiet! Going from 2 disks in mirror hammering around to a quiet SSD-disk was just surreal.
- Applications start immediately. Even heavy stuff like Outlook with addons and ~5GB PST-file takes just a couple of seconds.
- Chrome and IE starts immediately. Web pages with Flash load much faster.
- Start menu searching (even with sh*tloads of mails and files indexed) is done faster than I can type each letter.
- Windows now feels like this was what Microsoft intended Windows to feel like. Very responsive!
- My old drives could take about 65G of shock, the SSD can take 1500G. Perfect for laptops!
What changed?
- Data transfer speed is increased by around 10x.
- Seek time is gone. In a HDD the heads needs to keep moving back and forth to do read/write operations. This is a damn slow process. Access to any file requires the heads to be moved to a certain location.
- Disk fragmentation is (almost?) history. I haven’t looked into the details yet, but since NTFS default cluster size is 4k, Windows has 4k page size (memory blocks) the SSD disk works on 4k blocks it’s a pretty good match. I’m guessing there is some penalty of fragmentation, but it is nowhere near that of a HDD.
- Even though I have enough memory I noticed an increase in speed on virtual machines running from a normal HDD. Clearly even if Windows has enough memory it is using swap/page file, so having the page file on a SDD disk is very beneficial.
- Kernel (and applications) no longer has to block for a relatively long time while waiting for IO. This is a problem with HDD because even if there are free resources on the system the whole process will hang while waiting for IO. When the wait is lower the app is much more responsive.
- I chose to buy a disk which was fast in read and write (285MB/s read and 275MB/s write). There are some disks that are faster in read and much slower in write. For most users the slow write won’t be noticed because of Windows write cache so I don’t think its necessary to put too much weight on write speed as long as it is ok. 90% of disk IO is probably read anyway.
Disadvantage
So is it all good?
- The 60GB drive I bought shows up as 51GB.
- After installing Windows 7 Ultimate + Windows Update, Office 2010 Professional, Visual Studio 2010 Ultimate, setting up email (a few GB of data) and installing the usual utils (a few browsers, 7-zip, notepad++, paint.net, daemon tools, etc) I was left with 7,5GB free space. Even not counting Visual Studio and stuff there is less than 30GB free. I would guess I need a 100-120GB SSD to be fully happy. But that is not a problem. Another 60GB disk mapped as "C:\Program Files” on the first disk will only increase speed even further. Do I dear run it in stripe?
- It is expensive. You get 1500GB HDD for the price of 50GB SSD.
Now the question is this: Are SSD disks safe enough to run in RAID STRIPE? 
.Net GUI thread crossing
What is the GUI thread?
- Update the GUI.
Make changes in GUI objects like changing Text in a TextBox. - Read Windows Message Queue.
The WMQ is where Windows put all messages to the application like key clicked, mouse clicked, mouse moved, etc. - Repaint the GUI.
Taking the application from “code” to “graphics”.
The solution?
Timers
Do(nt)Events
Invoking GUI thread
private delegate void GUIInvokeMethodDelegate(Action @delegate); /// <summary> /// Invoke command with GUI thread. Usage: GUIInvoke(() => FormOrControl.Cmd()); /// </summary> /// <param name="delegate">Command to execute in form: () => Cmd()</param> public void GUIInvokeMethod(Action @delegate) { // Check if we need to invoke as GUI thread if (this.InvokeRequired) { this.Invoke(new GUIInvokeMethodDelegate(GUIInvokeMethod), @delegate); return; } // Execute @delegate.Invoke(); } // Sample method for invoking some code as GUI-thread public void DoThisAsGUI() { // Invoke some code (anonymous method) GUIInvokeMethod(() => { // Something you want to do in GUI thread. }); // Alternatively invoke a method: GUIInvokeMethod(() => SomeMethodThatRequiresGUIThread()); }
Disable Windows 7 Sound Card Jack Sensing
Windows attempts to auto detect any changes to the plugs in your sound card. That doesn’t work well with my sound card (X-Fi something) so the speakers keep falling in and out randomly.
Solution:
The sound card drivers are located under the registry key:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Class\{4D36E96C-E325-11CE-BFC1-08002BE10318}
Search for “EnableDynamicDevices” (located under GlobalSettings somewhere) and set it to 0 (zero).
Note: You may have multiple sound cards (and old drivers), so repeat search to find all occurences in ClassId branch described above.
I’m spent a few days in figuring this out so I hope it helps!
Edit: Yes! The sound card on my Lenovo W510 seemed really bad (on powerfull high pitched sounds it would “break through” and play music normally). I figured since the W510 had only 1 socket it was a combined speaker/mic. So I did above patch, rebooted and suddenly the card works as expected.
Slow YouTube
For the past 12-18 months YouTube has been slowing down dramatically for me. Around 18:00 I have to pause videos to let them download ahead, and around 20:00 there is no point in even trying.
So I ran some speed tests that showed my ISP (www.get.no) to be throttling the bandwidth for YouTube. I’m not sure how accurate these tests are. Anyway I fired off an email to my ISP asking why, and the reply was that the Scandinavian YouTube caching servers were overloaded.
From that I deducted that if YouTube has different servers for different regions (makes sense) they would either have to break internet routing standards so they can use one IP in several regions, have one gigantic NLB (Network Load Balancer) or they would simply spread the load using an intelligent DNS server.
Surely enough after having someone in the US do a DNS lookup I got confirmed that the DNS will round robin only a certain segment of the IP’s for me, and a whole different segment for the person in the US.
Solution?
I added the US YouTube IP to my local hosts file. This has increased YouTube to the point where I’m able to see 320/480 videos without pausing even during rush hour.
Another solution which I’m testing now comes from http://www.videoaccelerator.com/ – I’m trying out the free version.
Enabling Windows Firewall on Windows 2003 Active Directory Domain Controller
Some time ago I discovered that my (two) Active Directory controllers weren’t actually talking to each other too well. After some debugging I found that the local Windows Firewall was enabled on one of the AD controllers and it was basically blocking everything that had to do with AD. So I disabled it and all was well. All servers are located behind a firewall anyway, so the risk was minimal.
Now I thought it was about time to enable the firewall on my AD controllers again. And it turns out that, as usual, Microsoft lets the ports flow wild. There are three services that don’t have a fixed port: NTDS, NTFrs and NetLogon. Without a fixed port it is difficult to enable firewall for them. The local firewall can enable based on process, but my external can’t.
The good news is that you can set fixed ports for these services. This is described in the knowledge base article http://support.microsoft.com/kb/555381.
But who wants to read all that?
Here is the short version:
- Change registry to set fixed ports either by follow the knowledge base article or simply by running this .reg-file.
Note: As always when downloading .reg-files have a look at it first to see what it actually does.
Download, unzip, execute .reg-file, answer yes to question about importing it: - Reboot the AD-controller.
- Note that if all servers are on same subnet you can increase security a lot by setting Scope to “My network (subnet) only”.
Enable Windows Firewall and set the following rules:
- Repeat on all AD controllers you want to enable firewall on.
- Make sure you check Event Log on other servers for errors related to enabling firewall.
| Port | Protocol | Name | Notes |
| File and Printer Sharing | Already exist, must be enabled. NOTE: If servers are not on local subnet you may need to modify Scope. |
||
| 53 | TCP | DNS (TCP) | |
| 53 | UDP | DNS (UDP) | |
| 88 | TCP | Kerberos (TCP) | |
| 88 | UDP | Kerberos (UDP) | |
| 123 | UDP | NTP | |
| 389 | TCP | LDAP (TCP) | |
| 389 | UDP | LDAP (UDP) | |
| 3268 | TCP | Global Catalog LDAP | |
| 53211 | TCP | AD Replication | This was set by the .reg-file |
| 53212 | TCP | File Replication Service | This was set by the .reg-file |
| 53213 | TCP | NetLogon | This was set by the .reg-file |
| Remote Desktop | Optional: Recommended/required if you access server remotely. |
Enjoy! 
Tip: Upgrading VS2008 projects to VS2010
Many of us are upgrading our projects to Visual Studio 2010 nowadays. You start Visual Studio 2010, load the project and the conversion wizard pops up. You click finish, try to compile and get a bunch of errors. Oh no!
The fact is that what Visual Studio version you are using is totally irrelevant to the code. If you are using .Net 2.0, 3.0 or 3.5 then the code should not change and Visual Studio 2010 should be able to execute it. But it doesn’t… Why? Because the conversion wizard is defect. Actually only the solution and project files needs to be changed, but the wizard often attempts to change a whole lot of code as well.
The solution is very simple: Convert the solution, but only keep the new solution and project files.
Step by step guide:
- Copy your project to a new folder.
- Open the new copy in VS2010 and let VS2010 convert it for you.
- Close VS2010.
- Copy all converted solution and project files to their corresponding place in the original folder. Files are *.sln, *.csproj, *.vbproj.
- Now open the original folder in VS2010 and everything will work just fine.
Alternatively if you are connected to a source repository like TFS, SVN or GIT then you can use the original code folder and then undo changes to all files except solution and project files.
BitLocker startup error 0×26000070
I’m using BitLocker for HD encryption on all partitions, USB disks and USB pens. For USB* it works fine by just asking for a password. But for HD encryption it is a bit more complicated.
Basically BitLocker should encrypt your HD without you having to provide any password at every startup. It should also be secure from tampering because if anything changes on the PC then it looses the key and you have to provide a recovery key/USB-pen.
Since I’m using a docking station for my laptop it kept asking me for recovery key at almost every reboot. I quickly discovered that this was related to the BIOS BOOT ORDER, where the docking station would sneak in some new devices. Simply telling BIOS that I only want to boot from HD fixed that.
Later I was so smart that I tampered with boot settings (BCD), and now I’m getting a 0×26000070 at almost every boot. The great internet contains very little information about this, but I managed with some lucky guessing to track it down to some PCI Express settings during boot. Something about using BIOS or OS IRQ settings?
Anyway, if you have this problem then try this: BCDEDIT /SET usefirmwarepcisettings false
BCDEdit /enum should list it as No.
Give it a couple of reboots and let me know how it went…