Skip to content

Customizing Windows 7’s logon screen and user picture with Group Policy

 

How can we get our logo on here?

Is what my boss’s boss asked me right after I installed Windows 7 on his machine. Being one of the first 5 people in the office with a Windows 7 workstation and a pioneer in my rollout, I quickly told him: of course! … then I got to work on the best way to accomplish that.

Prep work

The first thing I did was create a location on my network accessible by all authenticated users, this is where the custom artwork will live.

The user picture

Is a 128×128 pixel bitmap, which our logo happens to fit nicely into, so I grabbed the graphical part of our logo and shrunk it to fit into a square 128×128 pixel bitmap, saved it to its home on the network and moved on to the background images.

The background images

Windows 7 supports multiple files, a default file called backgroundDefault.jpg and 12 other files with the resolutions appended to the default name (ie. background1920x1200.jpg). The system will determine which file to use (if the file with your screen resolution exists) in %windir%\system32\oobe\info\backgrounds (can be created if it doesn’t exist). If you omit an explicitly sized file for a screen resolution, the default file will be stretched to fit your resolution. One final note, images must be less than 256kb in size. The list of supported resolutions is below:

background768x1280.jpg
background900x1440.jpg
background960x1280.jpg
background1024x1280.jpg
background1280x1024.jpg
background1024x768.jpg
background1280x960.jpg
background1600x1200.jpg
background1440x900.jpg
background1920x1200.jpg
background1280x768.jpg
backgroundDefault.jpg

 

Once I had the background images for the resolutions that we have in the office, I saved those files to a subfolder in the network location I created previously and launched the Group Policy Management console.

Group Policies

Now that we have our resource files on the network we just need to tell Group Policy what to do.

Create a new Group Policy Object in an OU that contains the computers you want to customize and make the following changes to the policy.

From the Computer Configuration –> Policies –> Administrative Templates: enable Apply the default user logon picture to all users (from Control Panel/User Accounts) and enable Always use custom logon background (from System/Logon).

From Computer Configuration –> Preferences –> Windows Settings –> Files: add a new File item with a target path of %programdata%\Microsoft\User Account Pictures\user.bmp, from General make the source the 128×128 user picture we saved on the network (use its UNC path), you can leave everything else with its default setting but make its Action Replace.

From Computer Configuration –> Preferences –> Windows Settings –> Files: add a new File item with a target path of %systemroot%\System32\oobe\info\backgrounds\ (make note of the trailing \, it is required to indicate the target is a directory, not a file), from General set the source file(s) to be the directory we saved (use its UNC path) but append \* to the filepath, to indicate that you want to copy all the files from this subdirectory to the target. As with the previous item, you can leave everything else with its default values but make its Action Replace.

From User Configuration, leave everything blank.

Once done, your output should look like the screenshots below (replacing \\domain.local with your own domain’s FQDN):

 

 

Further reading: Rafael Rivera’s Within Windows post was a valuable point of reference for this project.

Saved Windows Credentials on Windows 7 Home Premium

The problem

I have a few Windows 7 Home Premium machines at home which I network together. I don’t use HomeGroups yet, and may not in the near future but I do keep some resources on the network that each machine has to access. Windows 7 Home Premium (and lower) does not remember credentials for other Windows machines (if you click the “Remember this password” checkbox in many authentication dialogs, Windows will not remember them and re-use them on next logon) so this becomes a problem when you are using service accounts to access shared media across machines (I don’t have an Active Directory domain at home).

The workaround/solution

My old solution was to create a service account on the machines needing to access the shared resources and hide the service account from view on each machine.

My new solution is to use the Windows Credentials Manager (in the Start Menu, search for “Manage Windows Credentials”). This will bring you to the Operating System’s Credential Manager, where you can Add a Windows credential. From this window, type in the server name and local username and password that authenticates on that server (like your service account) and click on OK.

Windows will now use those credentials whenever it tries to authenticate against that machine.

All you need to do now is to remember to change the password here whenever you change it on your servers for your service accounts.

Remember to check the MAC address

Today I encountered a weird issue with a new machine I was setting up. The NIC on the machine worked to PXE boot and install Windows XP off the WDS server at work, however once I installed the drivers for the NIC on Windows XP and obtained an IP address from DHCP I couldn’t reach anything.

I couldn’t ping my default gateway, any DNS servers, couldn’t ARP an IP address, nothing. I tried to release and renew the IP address, ran diagnostics (which all passed except the connection diagnostic) and I was getting an IP address on the correct subnet and with correct values for the gateway and DNS servers. I was stumped!

I knew the NIC was working because I’d installed the OS through the network, but even though the NIC was sending packets out, none were being received (as reported by the OS). I called my boss and started throwing ideas back and forth, I’d already done most of what he was mentioning except one thing: Look at the MAC address that the NIC is reporting. It was 00-00-00-00-00-00.

I completely skipped over that the first time I ran an ipconfig /all on the machine, but as soon as I saw the MAC address contained all zeroes I quickly went to the properties page that installed with the NIC driver, went to the MAC masquerading tab and clicked on “Use default” – nothing changed. The MAC address field was empty and while I could have maybe retrieved the MAC address off the NIC itself or motherboard, in the interest of time my boss found the vendor ID for a company I’d never heard of (maybe he’d used them before, who knows, but we didn’t have any of those adapters in the office) and gave me the first 3 pairs belonging to that vendor for the MAC address. I set the last three pairs to random numbers and seconds later after an ipconfig/release && ipconfig/renew I was finally able to send and receive traffic on the network. Ideally you’d try to retrieve the MAC address that belongs to the NIC, however, we manage the environment and know what devices come into the network so it is very unlikely that we will encounter any conflicts. No excuse, I know, but that’s just the way things are right now.

Neither of us had seen something like this before, but the advice to take with you from this post is: remember to check the MAC address if you ever encounter a similar issue.

Remember to check the physical wireless switch

On a related note, this brought back memories of a colleague of mine who while working on one of my old employer’s help desks took a  call once, minutes before the end of his shift. The user was reporting that his wireless connection was not working and my colleague spent the next 45 minutes troubleshooting with the user only to find that the problem was resolved by toggling the physical wireless switch on the laptop to the On position. Yes, that simple.

Details matter, remember this.

Know about WinRM (Remote Management) and WinRS (Remote Shell)

 

I just recently migrated my data to another machine (to use as a file server) since it has a Sans Digital NAS attached to it. Normally working with files across the network (I now map a shared drive to the share on the server) is perfectly acceptable – however, there are some operations better left on the remote machine, for instance: extracting or compressing [rar/zip] archives.

Ideally, I’d prefer to have a solution where I can (in this case) right click on an item on my machine and run a command on the remote machine, and with WinRS and some programming I may get it done eventually, but in the meantime as a quick and dirty solution I set up WinRM on both machines and I currently start a command prompt instance using WinRS and I can use the command line rar.exe or 7za.exe for dealing with archives on the remote machine.

TechNet has more information on WinRM and WinRS, for brevity I am just going to outline what I had to do to get my current solution working:

First, from an administrative [read: elevated, if applicable] command prompt execute: winrm quickconfig on both machines, answering each prompt with yes.

If the machines are domain joined then they already trust each other thanks to Kerberos, however if the machines are part of a workgroup you need to tell each machine to trust the other by executing: winrm set winrm/config/client @{TrustedHosts=”%remotecomputername%”}. (Note: replace %remotecomputername% with the hostname of the remote computer.)

Now you should have all that is needed to either remotely manage each machine from the other using winrm (e.g. starting or stopping a service, or querying information using WMI) or run commands on the remote machine using winrs. My particular usage is: winrs –r:%remotecomputername% cmd.exe. This command will start a remote command prompt instance that I can then use to navigate directories on the server and run command line utilities like 7za.exe or netsh, ipconfig, etc.

If you prefer PowerShell, there is one more step that needs to be taken from within an elevated PowerShell command prompt, on the machine that you want to remotely connect to execute: Enable-PSRemoting. This will enable the PowerShell listener and PowerShell should then accept connections from remote PowerShell instances. To connect to the remote PowerShell session execute: Enter-PSSession %remotecomputername%.

Again, I did not go into any detail on this because 1) I am not an authority and I didn’t look further than this, and 2) there is plenty of information on the web and TechNet regarding this topic.

I will probably post an update to this post if/when I ever decide and have enough time to write some code that will allow me to execute canned commands on a remote machine (mainly targeted at extracting archives on the remote machine right now).

Enjoy reading up on WinRM/WinRS.

Indexing additional mailboxes in Outlook 2007

I recently came across an issue where an administrative assistant needed to search her boss’s huge mailbox while it was attached in Online mode to her Outlook profile. Normally, Outlook search will work for certain folders (like Calendar and Contacts) because it caches those folders from attached mailboxes so that they can be indexed; however in this case the amount of items in each of those folders was either too large or something else was going on and search just wasn’t working.

After searching online, I was able to find out about the CacheOthersMail registry value which can be set in HKLM\Software\Microsoft\Office\12.0\Outlook\Cached Mode and is looked for by Outlook after a post Office 2007 SP1 hot fix (which is included in Office 2007 SP2). This value will instruct Outlook to cache any attached mailboxes in the user’s OST file – this being useful because it increases the access speed of the mailbox significantly and I thought should allow the indexer to index the mailbox. Much to my chagrin, I found that the mailbox was still not indexed after creating this value in the proper registry key (but I could confirm that the mailbox was indeed cached); I had to continue looking.

It was then that I was pointed to the following (excellent) blog post by Mike Lagase: Windows Desktop Search Indexing in Outlook. This blog post detailed an interesting setting (which could be potentially dangerous when used improperly): Enable Indexing of Delegate Mailboxes. This setting, along with other useful Group Policy settings for Windows Desktop Search 4 when a user needs to index mailbox contents are discussed in Mike’s blog post.

Make sure to read the blog post for more details, however my fix was to set the CacheOthersMail  registry value along with enabling the Enable Indexing of Delegate Mailboxes policy. The end result of this was Windows Desktop Search 4 indexing the contents of the already cached attached mailbox.

The administrative assistant is now happy again that she can search her boss’s mail like she was able to in the past.

Migrating to Google Apps (part 4)

Google Apps with Exchange ActiveSync

I am an iPhone user, having adopted the device with the iPhone 3G (as soon as Exchange ActiveSync support was added). Needless to say, I depend on the excellent ActiveSync protocol for push email, calendar and contact synchronization. I think I started considering Google Apps a viable solution for my personal email when I learned that Google had licensed this technology from Microsoft.

I have setup my mail account on my iPhone with minimal issues, following Google’s instructions. Everything works the way you would expect, and while I have noticed that some folders sometimes take a few minutes to reflect certain changes (only when I moved messages with more than one label, I think), everything else is quick and solid.

 

My messages aren’t being deleted!

After setting up the iPhone with my new Google Apps account I continued testing (mainly to confirm that all the Outlook issues from part 3 in this series were resolved) by sending myself messages and acting on those messages. For one or two test messages I deleted them from my phone without marking them as read. A few minutes later while looking at the web interface I see two unread messages in my “All Mail” label, when I clicked on the label I became confused. Both messages that I had previously deleted from my iPhone had apparently been archived instead of moved to the Trash. I started searching online.

It turns out that Google sync implements this as a “feature” to lock mobile users into not deleting messages and archiving them instead. I thought, “OK, how do I turn this off?” and found much to my chagrin that you couldn’t! This is one of the things I dislike about Apple (more on that in another post), I do not like anybody making decisions for me, especially when they affect the way I organize my mailbox.

 

The workaround

After coming to terms with the fact that I couldn’t do anything about this “feature” Google built into their sync product except complain with a post in their message forums along with many others, all asking Google to fix this soon. I started looking for a way to work around this annoyance. I personally never archive anything and all the messages in my mailbox are labeled, so this gave me the idea:

Since I don’t archive any messages without a label, I just need to identify archived message without any labels and that should yield any messages that I deleted from my iPhone that Google decided  NOT to delete.

With this thought I set off to search for a way to identify archived mail without labels. When I found out that there is no way to identify unlabeled archived messages I started searching for another way to accomplish this and finally hit with someone who had the right idea. The person’s suggestion was a dirty hack, but it was a solution: create a search that contains a “–label:” parameter for each of my labels, including the inbox, sent, etc. I have over 30 labels right now, so it took me a few minutes, but I did it, and sure enough there were the messages that I had deleted from my iPhone!

Of course, this method only works if you always archive your mail with a label (which I do). Additionally, the search needs to be updated if there is a change to any of the labels in your mailbox. However, I think this is acceptable for now because I don’t change my labels every day and I really hope Google fixes this annoying issue soon.

Now that I had a working search yielding desirable results, I just needed a quick way to access it from anywhere. There are Firefox plug-ins for this, such as Better Gmail 2 which includes a Persistent Searches script created by Mihai Parparita (requires Greasemonkey to run on Firefox). However, this method required 1) Firefox and 2) That I have the plug-in and Greasemonkey installed. I really didn’t want to be tied down to either so I kept looking. Finally, I found someone who had the right idea, create the saved search as a bookmark. This worked well but it still required me to have the bookmark on any machine I wanted to use the search on, which could be any machine if I felt like cleaning out messages I had deleted from my iPhone then and there from the web.

I finally decided on a compromise which is to create a directory on one of my domains containing an index.htm file whose sole purpose was to redirect the browser to my Google Apps Mail account with the search in the query string (using an meta refresh). As long as I was logged in (or logged in while navigating to the link) I would be taken to a search results page with all my “–label:” search parameters already applied, providing me with a list of the messages ready for deletion.

 

Summary

All things considered, Google Apps Premier is a great product and I don’t regret making the switch or giving them my money. I hope the product continues to mature and that Google fixes some of the annoyances I had to overcome in order to achieve a decent workspace. Hopefully this series will serve someone other than myself should I or they have to set this up for anyone else in a similar manner.

Migrating to Google Apps (part 3)

Outlook issue: duplicates everywhere

Once the Google Sync tool completes the first synchronization of the mailbox you will see a replica of your Google Mail inbox in the mail folders pane in Outlook. Aside from the built-in Outlook folders (of which Inbox, Sent Items, and Deleted Items are mapped to Inbox, Sent Mail, and Trash respectively in the cloud, there is also a new [Archived] folder created in Outlook mapping to All Mail online) you will see each of your Google Mail labels show up as a folder in the Outlook folder list. The Contacts and Calendar will also be synchronized with their counterparts in the cloud, all is well so far, until you start noticing duplicate messages.

Knowing that each label is equivalent to a folder is not a problem if you don’t use search folders (like Unread Mail) because you will see each message marked as unread in its folder at a time (read: a message with three labels will show as unread in three folders) and once you read the message it be marked read in every folder it belongs to automatically.

The problem for me is that I take advantage of search folders, specifically Unread Mail and For Follow Up to help me keep track of messages without having to switch from folder to folder. This is a problem because if, for example I sent myself an email and then marked it for follow up, it would show up twice in my For Follow Up search folder! Once for the message in the Sent Items folder and once for the message in the Inbox.

Another similar issue is when I send myself an email, before I mark the message as read it will show up as unread in the Sent Items folder as well as the Inbox. This too, was unacceptable because I am constantly sending myself reminders when I’m not near OneNote.

The final thing that bothered me was the task list in the To-Do Bar as it mimicked the For Follow Up search folder’s behavior showing multiple flags depending on where the messages were in the mailbox (ex. two flags for a message in Sent Items and the Inbox that was flagged for follow up.)

I tried to live with it for 5 minutes, then decided I needed to make some changes to mitigate this annoying side effect of the way Google decided Outlook should behave when dealing with labels. What I ended up doing was creating custom search folders to mimic the built-in Unread Mail and For Follow Up search folders and customized the Filter property of the task list view in the To-Do Bar.

Below are the criteria and display options I chose for the two search folders and the task list view.

 

Create your own custom search folders

I deleted the built-in Unread Mail search folder, for some reason the For Follow Up folder wasn’t created in the list of “Search Folders” when the Google Apps account was created, so that was one less search folder to delete.

I then proceeded to create my new custom search folders, starting with the Unread Mail search folder:

  1. Right click on the “Search Folders” item in the mail pane and select “New Search Folder…”.
  2. When presented with a dialog asking for the type of search folder you want to create, go all the way down to “Create a custom Search Folder” and click on “Choose…” to choose the criteria.
  3. In the “Custom Search Folder” dialog name the new search folder “Unread Mail” (make sure to delete the built-in search folder to avoid a name conflict) and click on “Criteria…”.
  4. When presented with the “Search Folder Criteria” dialog, click on the “More Choices” tab, then click on the “Only items that are:” checkbox, and make sure that the drop down reads “unread”.
  5. Click on the “Advanced” tab, then click on the “Field” drop down, navigate to the “All Mail Items” sub menu and click on “In Folder” from the menu that opens up.
  6. Set the condition to “doesn’t contain” and the value to “Sent Items” and click on “Add to List”.
  7. Click on OK twice to go back to the main Outlook window. The search folder just created should be in the “Search Folders” section of the mail pane.
  8. Right click the Unread Mail search folder and select “Add to Favorite Folders” to show the search folder in the “Favorite Folders” section of the mail pane (all the way at the top).
  9. NOTE: This method will not remove any duplicates resulting from multiple label assignments in Google Mail, you will still see duplicate messages in the Unread Mail search folder if a message that is unread has been tagged with multiple labels in Google Mail.

The procedure to create the custom For Follow Up search folder is similar, see below:

  1. Follow steps 1, 2 and 3 above, but name the search folder “For Follow Up” instead of “Unread Mail” and click on “Criteria…”
  2. When presented with the “Search Folder Criteria” dialog, click on the “Advanced” tab.
  3. Click on the “Field” drop down, navigate to the “All Mail Items” sub menu and click on “In Folder” from the menu that opens up. Set the condition to “doesn’t contain” and the value to “Sent Items” and click on “Add to List”.
  4. Click on the “Field” drop down, navigate to the “All Mail Items” sub menu and click on “Flag Status” from the menu that opens up. Set the condition to “exists” and click on “Add to List”.
  5. Click on the “Field” drop down, navigate to the “All Mail Items” sub menu and click on “Flag Completed Date” from the menu that opens up. Set the condition to “does not exist” and click on “Add to List”.
  6. Follow steps 7 and 8 above, right clicking the For Follow Up search folder instead of Unread Mail to add to favorite folders.
  7. One additional step to completely mimic the built in For Follow Up search folder is to right click on it (from the “Favorite Folders” section or under the “Search Folders” section in the mail pane) and click on “Properties”, then in the properties dialog for the search folder about halfway down the dialog make sure the “Show total number of items” radio button is selected as opposed to the default “Show number of unread items” and click OK.
  8. NOTE: This method will not remove any duplicates resulting from multiple label assignments in Google Mail, you will still see duplicate messages in the For Follow Up search folder if a message that is flagged has been tagged with multiple labels in Google Mail.

One thing you should know about the search method above is that if you have messages which had been flagged for recipients or show with a flag status represented with that icon previously and they did not show up in the default For Follow Up search folder, they will show up now. The corrective action I took on these messages is to search for all flagged messages in “All Mail Items” (from Outlook 2007) and for each message that was flagged for recipient or had that icon I right clicked on them and from the “Follow Up” menu, clicked on “Clear Flag/Delete Task” (or similar, I don’t quite remember). This only kept the messages I had flagged for follow up myself (identifiable by the sole Flag icon) in my For Follow Up search folder.

 

Modify the view of the task list in the To-Do Bar

The final thing that was bothering me was that if there were any items (like messages I had sent to myself) that I had flagged, they would show up as duplicate items in the To-Do Bar’s task list. I had fixed the issues above by creating my own search folders, but this view wasn’t created using a search folder. Instead, it uses a filter, which can easily be modified by following the steps below:

  1. Right click on any empty space in the task list and click on “Customize Current View…”.
  2. In the “Customize View: To-Do List” dialog, click on the “Filter…” button.
  3. In the “Filter” dialog, click on the “Advanced” tab.
  4. This should look familiar, click on the “Field” drop down, navigate to the “All Task Fields” sub menu and click on “In Folder” from the menu that opens up. Set the condition to “doesn’t contain” and the value to “Sent Items” and click on “Add to List”.
  5. Click OK twice to go back to the main Outlook window. You should no longer have duplicate task items for any messages that you have sent to yourself.
  6. NOTE: This method will not remove any duplicates resulting from multiple label assignments in Google Mail, you will still see duplicate messages in the To-Do Bar task list if a message that is flagged has been tagged with multiple labels in Google Mail.

 

Additional considerations

It is important to remember that Outlook works with folders as while Google Mail works with labels, in my opinion labels are vastly superior to a traditional folder hierarchy, but you have to use what is there. With this knowledge, you should be tolerant and try to avoid assigning multiple labels to a message if you normally work in Outlook and rely on search folder functionality as you will often see duplicate messages in your views if you don’t, possibly throwing off your counts.

This following statement is my assumption based on my observations, feel free to correct me if I’m wrong: Google Sync takes care of making sure that each labeled message is physically located in each folder on the PST file that locally holds your mailbox, so the PST file will be larger than your mailbox size as reported by Google Mail due to this.

As a side effect of my statement above, whenever a message is unread, or when a message is flagged, if it has more than one label assigned to it the message will show up more than once in your search folders, and Outlook folder in general. A big problem with this is sent items. When you send a message to yourself using Outlook, the message will arrive as “read”. I have only seen this behavior when a message is sent through Outlook, sending yourself a message from the web or a mobile device keeps the message unread as you would come to expect. The reason for Outlook’s behavior in my opinion is that the Google Apps Sync tool discards messages that would traditionally be copied to the Sent Items folder is deleted and instead is replaced with a message with the Sent label (or the message itself is applied with a sent label) thus moving it to the Sent Items folder. Google goes out of its way to ensure that the state of messages is kept intact for everything that has a label in Google Mail. A sent message is tagged with the Sent label, and if it arrives in your inbox it is also tagged with the Inbox label. So any message you send yourself always has more than one label, and by Google’s reasoning has to be kept in sync. This is why my custom search folders above worked for the messages I had sent myself (as reminders to myself which were subsequently flagged), there were no duplicates because I effectively filtered out the Sent label within Outlook by excluding the Sent Items folder from the searches.

So far, I feel content with the migration on the desktop side. My next and final post in this series will detail the steps I had to take to get around a ridiculous Google mobile sync “feature” that annoys many iPhone (if not mobile) users, including myself.

Migrating to Google Apps (part 2)

Migration pains

The Google Apps Sync tool for Microsoft Outlook creates a new Outlook profile so that you can keep any other accounts separate from your Google Apps account (Exchange promotes this type of profile use as well). This tool is what eventually keeps your mail, contacts and calendar synchronized between Outlook and the cloud.

The sync tool also provides the ability to import your existing Microsoft Exchange mailbox to your Google Apps account, I naively accepted Google Apps Sync’s offer to do this and let it get to work. The tool successfully migrated my calendar and contacts to their counterparts in the cloud, it created labels replicating my folder structure in Exchange and started uploading my mail messages to the cloud (while showing them in Outlook as it worked).

I started going through the messages and noticed that my name was missing in the To and From fields of my messages, depending on who the sender and recipient was. This was unacceptable and I thought I might have been doing something wrong or that there was a bug with the tool. It turns out that there is a bug with the tool (Outlook/Exchange may be to blame, who knows) where the aforementioned fields were not preserved for any messages in your mailbox if the sender or recipient was a member of your Exchange organization. I can only guess that this is because Exchange/Outlook strip out this information and replaces it with the unique contact information from the GAL. Either way, this was unacceptable and I quickly aborted the synchronization process, cleaned up that newly created profile and all messages from the Google Mail account and started searching for an alternative way to import (namely, IMAP) since I read that this was one of the methods for importing mail that preserved this information (as well as time/date information for messages).

The Google Apps Advanced tools section of the dashboard provides a section where you can provide account details for one (or many) users, a root folder to start importing from in the mailbox and a list of folders to exclude (for example: /Contacts, /Calendar, /Sync Issues) from the migration. The migration tool will then log into your mailbox through IMAP and download all the messages in the folders found under the root folder specified. When specifying a root folder using / as the root folder is usually enough (however, if you use / as the root, you will want to exclude /Contacts and /Calendar because the tool will import these items as mail), but for example if you wanted to migrate only messages found in the Inbox and its subfolders you could specify /Inbox as the root folder and specify any folder exclusions in a relative path to the root folder that you don’t want synchronized, like /Do not bring along if this folder is a subfolder of the Inbox.

Now, as long as your server is configured for IMAP and you have provided the settings (root folder and exclusions, among other options that are pretty self explanatory) you can kick off the migration tool and it will log into the server and download the messages as per your specifications. This process can reportedly be slow but since my mailbox wasn’t huge and it was only working on my mailbox, I left it for a few hours and when I sat back down the process had completed.

The results of the migration were impressive, the mailbox was almost completely there: the sender/recipient names, message times and dates were all correct (save for a few probably malformed messages that may have been sent from the future). I found that the only messages that were not migrated were messages with attachments that Google Mail does not support (such as executable attachments and zipped executables, etc), or attachments which had a size (including attachments) >25MB. Here is an overview of the IMAP migration process. The messages that were not migrated messages were very simple to filter and once I confirmed that I didn’t need them (since I would have the backup PST of my mailbox as it was on the Exchange server to reference these messages if I ever needed them anyway) I deleted those messages as well as the messages with invalid date information since I didn’t want them to sit out of order.

Finally, the tool created a label for each folder in my folder structure and tagged each message it found with its corresponding label (thus maintaining my folder structure to be displayed by Outlook and other clients that treated labels as folders such as the iPhone’s Mail.app). Another nice touch is that every imported message was labeled with the “Migrated” label so that you could easily differentiate messages that arrived while the migration was taking place and messages coming in from the Exchange migration.

 

Additional considerations

While in the end I achieved the results I was looking for, I feel that I had to unnecessarily jump through hoops to properly get my messages into my Google Mail account. The fact that I had to use the IMAP migration tool was also inconvenient since one of the features on the Google Apps Sync tool is that it facilitated client-side migrations to Google Mail. While the IMAP tool is infinitely useful for large migrations, small shops might prefer to do client-side migrations, not to mention it is the easier approach. The Google Apps Sync tool should have been able to query the GAL (if available) and fill in the information missing from the messages before sending them upstream to Google’s servers.

It should also be noted that if you have more than three or four nested folders in your mailbox unless the names of the folders are very short you will likely exceed Gmail’s 40 character label limit. The migration tool I assume has a special pass to bypass this limit for the sake of maintaining the original folder structure intact, however don’t expect that you will be able to edit this label (or create labels to nest deeper) from the Gmail interface. I didn’t test if you could create nested folders using Outlook (and have the Sync Apps tool create the labels for you automatically online) instead I just opted to modify my folder structure to better embrace the label system, having labels for nested folders instead of the hierarchy except where it was absolutely necessary. This helped me stay below the limit and so far I can’t complain.

 

My next post will look into some of the challenges of using Outlook with the Google Mail way of doing things.

Migrating to Google Apps (part 1)

Why I made the decision

Google Apps is Google’s response to Microsoft Exchange environments. Each have their pros and cons, and honestly, in the workplace Google Apps is still playing catch-up to Exchange but I was looking for something I could use for my personal email account. Having access to all your emails, calendars and contacts in the cloud is almost a necessity, especially since we are constantly on the move and are utilizing our mobile devices more than ever before. If you are daring enough you can keep your documents online as well. If you spring out the $50/yr per user for Google Apps Premier you are additionally provided with additional services like [SSO to] Google Chat, Google Video, Postini, Google Sites and mobile device access for your Blackberry in addition to the iPhone and a few other platforms that Google’s Sync software supports. There are also other perks like Single Sign-On (SSO), forced SSL (configurable), your mailbox size is bumped up to 25GB per user, and they claim “Blackberry Support” which I can only assume includes Push email support since Google Sync only provides Contact and Calendar sync at the time of writing.

Everything I’d heard before about Google Apps while impressive still wasn’t enough for me to make a change, I was dependent on Outlook. When I learned about the Google Apps Sync tool (only available to Google Apps Premier and Education editions) is when I started giving the possibility of using Google Apps some serious consideration. In the end, I did some final research into the features I would be giving up on Exchange in favor of Google Apps Premier and finding them acceptable I signed up. The Apps Sync tool provides the user with Exchange-like access to Mail/Contacts/Calendar – changes are pushed and faster than IMAP for mail, and the Contact/Calendar synchronization is not far behind that as well. Your data is stored in a PST file and it just works – there are a few kinks relating to the way Outlook handles Gmail labels [as folders] where it duplicates any message that has more than one label in its corresponding folder and the email/follow up flag duplication you are likely to see, but I cover how I overcame these hurdles in a later post.

Now that I knew I wouldn’t have to give up Outlook on my main PC (a requirement for me since I was perfectly fine with using the Gmail interface everywhere else) I felt ready to make the jump, I signed up.

 

The sign up process

The sign up process as I remember it was really straightforward, the main requirement is that you own a domain name. You will be asked for billing information, to create an administrative user, some other details and finally be asked to confirm that you own your domain (either by uploading a file there or creating a CNAME record that Google provides you with). Once you confirm that you own the domain you are brought to the dashboard for the domain you just signed up with:

 

Everything is currently activated, however, when you first sign up, you need to manually activate Email and Postini Services by modifying your MX records from your domain’s DNS control panel. The mobile sync service (bottom right) also needs to be enabled by opting into it.

Google makes the configuration process fairly painless by providing a setup guide the first time you sign into the control panel which you don’t have to follow but if you need guidance it is invaluable.

Activating Email and Postini can take up to an hour each so you need to be patient while this process completes, in the meantime feel free to go through all the options offered to you by every other component of the Google Apps suite and configure them to your liking.

Additional Considerations

If you opt not to go through all the pages in the Google Apps dashboard at least be aware of the following:

  1. The Support tab provides you with a Customer PIN# and Support PIN# (not sure if it applies to Google Premier only) to contact support and bulk user creation.
  2. The Advanced tools tab provides you with many useful options such as: authentication options, reporting options, email migration.
  3. The Service settings dropdown (last link on the navigation menu) provides you with quick access to each of your service’s settings pages.
  4. It is wise to customize the URLs of your services so that you can access them without having to type out the Google Apps URL. For example: http://webmail.example.com instead of http://mail.google.com/a/example.com or http://cal.example.com instead of http://calendar.google.com/a/example.com. All you need to accomplish this is create CNAME records on your domain’s DNS control panel and let the Google Apps application you are changing know its new URL (so that Google can update its servers). The services will then be reachable via your custom CNAME and the long URL format. The URL change can easily be accomplished through the settings page of each service.

NOTE: I have not covered all the options for a Google Apps migration from Exchange, only the options that I chose to go with because they met my needs, this article has additional information regarding integration with Outlook and Exchange.

All things considered, the creation process and dashboard was very painless, the pain began once I attempted to get all my data into my account. I will touch on that in the next post.

Public Service Announcement: Make sure your backups are up to date

I am sure this has happened to more people than would care to admit it: Your clients’ (or your employer’s) backups are running smooth and you correct any hiccup right away but you are lazy about your own backups. So I start my (somewhat shameful) and hopefully eye opening post.

A few days ago I saw a Windows Delay Write Failed notification in my taskbar. I think most of us know it, this notification usually means a write operation failed, and if your [external] hard drive is plugged in, on, and should be working fine, you start to suspect that maybe the drive is failing or was somehow unplugged before data in the write queue was flushed. But I digress, the notification came up, I then proceeded to access the logical volume in question (which consists of a LaCie Big Disk spanning two 500GB disks on a RAID 0) and was able to access some of the files just fine. I then attempted to run a chkdsk on the drive but I was met with an error stating that the type of the file system was “RAW” and thus chkdsk was unable to scan it. This my friends, is when I started to panic.

Normally, I wouldn’t be worried because I knew that I had a backup from the night before, however, recently the hard drive (also 1TB) that I use to backup my data started malfunctioning culminating in an RMA. Once I received the new drive I procrastinated mainly because I was going to purchase a Sans Digital RAID enclosure which I planned on using as a new volume for all my data (consisting of a pair of 2TB disks on a RAID 1) soon. However, I was worried because I now had no viable backup of my data and for all I knew one of the two drives on the RAID 0 array had just gone bad.

I quickly prepared the replacement 1TB drive on one of my external SATA enclosures and prepared it to copy all my data using Robocopy over my Gigabit network (this would normally take ~12 hours for a full backup) but knowing that the file system was not being read properly I decided to try my luck in power cycling the LaCie Big Disk, boy was that a mistake! The drives wouldn’t spin up again.

I proceeded to “freak out” for about five minutes, actually having to take a few minutes away from my desk in order to get past the fact that I might have just lost 600GB worth of data. Once I was able to sit back down I proceeded to go through the basic troubleshooting steps:

  1. Take the enclosure apart (voiding the warranty, but at this point, I didn’t care about the warranty as much as my data)
  2. Re-seat the SATA power and data cables
  3. Disconnect each drive individually and see if the other would spin up

Everything I tried failed, things were looking grim.

Having another SATA enclosure on hand, I removed one of the SATA drives from the LaCie enclosure and installed it in my other enclosure, this time the drive spun up! I quickly confirmed the second drive spun up as well using the same method. By now at least I was sure that the drives were spinning so I started to feel some of the weight off my shoulders. My next step was to find out how to recover data from a broken LaCie RAID. Searching Google found a few noteworthy results and at least I was on my way to a real solution. While searching I heard about a few tools, R-Studio and iRecover being two of them, I would try them as soon as I had corrected my backup situation.

I also found the likely culprit for the problem: one of the blog posts I found online while searching stated that their problem was sourcing from the power adapter that the LaCie drive uses. The writer wrote that there was a ‘hissing’ noise emanating from the power adapter noticeable when your ear was placed on the adapter – I was able to reproduce this noise and confirmed that none of my other power bricks (albeit, different power bricks) were producing this sound when my ear was placed on them. The fix could have easily been to purchase a replacement power adapter, but this is the second Big Disk failure I’ve experienced so I will not be giving any more of my business to LaCie.

I spent the next 20 hours or so using WinImage 8.1 to create VHD images of each drive, the process would have been faster but I didn’t have enough space on the computer where I was making the image so I had to create the image and save to a network location (which I can only assume slowed the process down a little, though when I think about it maybe not so much as the USB bus since my Gigabit network should outperform the USB bus being used to read from the drive). Anyhow, after both VHD images were created I now felt that I had a viable backup of my drives and data. I put the two 500GB drives in a safe place and got to work on finding some tools that could “de-stripe” my array. Thanks to Windows 7’s built-in ability to Attach VHD files as local disks, I was able to perform all my tests on “Read Only” copies of the disks as if they were directly attached to the system (which was awesome).

The first tool I tried was Runtime Software’s RAID Reconstructor.  This tool was unable to detect the RAID parameters that LaCie used so in the end I couldn’t use it (and it wasn’t until later that I figured out why). The default detection settings that RAID Reconstructor used were out of the range of the RAID array start sector and the array block size was not one of the default settings the software provided you with (this LaCie drive used a 256KB block size, where the largest block size that RAID Reconstructor specified by default was 128KB). The parameters to search for the RAID array are fully customizable but I was unfamiliar with RAID parameters in general and the software’s RAID detection feature was unable to find the array parameters with the settings that I provided in addition to its own recommended settings so I uninstalled the trial and moved on to the next piece of software.

The next software I installed was iRecover by DYI Data Recovery. This software was easily able to find my array and reconstruct it, however the scan time was about 30-40 minutes. I was also unable to save the scan results so I would have to reconstruct the array every time (to be fair, it seems the registered version allows scan results to be saved). Once the array was reconstructed I was able to save a large file and confirm that it was not corrupted, great news! I was now ready to purchase iRecover, which costs $90, but first I wanted to try the last option that my search yielded: R-Studio.

R-Studio offers a powerful tool set for the technician with the R-Studio Local and R-Studio Network products. You can find out more about their products on their website. I downloaded R-Studio Local and got to work, I followed the RAID Recovery Presentation which provided me with enough information about the RAID parameters that I felt like I could do more than I could previously with RAID Reconstructor and within a few minutes was looking at my data, no scanning required! I will admit it took me a few tries to get the block size correct. I had a hint about the drive order (one drive would show as unallocated space while another came up as two partitions on the R-Studio GUI) so I used the drive order that I thought was correct at first (the drive with the partitions first, then the drive with unallocated space) and varied block sizes until I found the sweet spot: 256KB.

Armed with my tool of choice: R-Studio, I proceeded to recover the data from the two 500GB drives attached to my system via a pair of USB SATA enclosures (to avoid recovering from the VHD image across the network, though honestly I think my USB bus is taking a harder beating than my network would have taken) to a third 1TB disk attached via a third USB SATA enclosure. A few hours later my data is still being recovered but this goes to show that being lazy and procrastinating can bring you a lot more work in the long run.

I still haven’t purchased the Sans Digital enclosure I want, and while I will probably do that sometime next month, as soon as the data is recovered I will be re-scheduling my old Robocopy job to run every night and mirror one drive to the other over the network.

Never a dull moment.