First, a thank-you to Pierce from Logos Electromechanical who clued me in on what I was doing wrong with the backpack.

It turns out there are two common ways to detect an I2C device: address it and see if it ACKs (quick mode), or address it and try to read a byte from it (read mode). Some devices that aren’t set up to write don’t like to be read from and will cause problems if you try to do that (in this case, SDA gets pulled low and stays low!) I thought it was a bug but no, it’s actually how this device is supposed to work.

I selected an address for my backpack of 0×58. It turns out that i2cdetect has some intelligence when it comes to its default behavior – it will decide based on the address how best to perform its test of that address. If the device address is in the ranges {0×30:0×37,0×50:0x5f}, it will use read mode. Outside those ranges, it will use quick mode. See the i2cdetect.c code for how this was implemented in Linux.

So I chose… poorly – at least when it came to the address for the backpack (and keep in mind that the default address of this device is 0x7F, which is invalid for I2C, so you MUST select a valid address via solder bridging the appropriate pads!) Since the high (0×40) bit is already “1″ on this device, you can choose an address within the valid range of 0×40:0×77… I suggest something outside the 0×50:0x5F range though, to avoid accidentally triggering this (otherwise, be careful to use “-q” with i2cdetect). You can quickly tell when you screw up and try to read from this device: everything else will drop from your I2C bus and detect will act like every I2C address is in use by a device! (Unplugging the backpack from the circuit and from its power source for a moment will reset its state.)

You can read more about the I2C protocol here and here, learn about special addressing modes here, and see the specs here.

When using the Raspberry Pi model B note the I2C bus on the main header is numbered 1 (older boards have bus 0 on that header).

First, I snagged a BeagleBone Black kit. I haven’t had a chance to open it up yet but it looks promising, though I wonder just how much trouble it’s going to be… it sounds like it will be more daunting than my Raspberry Pi (which I’m finally making good use of!)

I also picked up a Logos Electromechanical Labs RGB LED Matrix Backpack to fiddle with. I’ve been busy getting I2C devices working with my Rpi and thought this would be a fun addition! (I’ve previously fiddled with them on the Arduino, but the Wire library on Arduino leaves much to be desired in terms of correctness). Sadly, while it works with the Arduino, the backpack appears to hose to I2C bus on the Rpi (note that since this backpack is a 5V device I am using an appropriate level shifter to interface it to the 3.3V Rpi). It seems that once it’s addressed (or even detected, e.g., via i2cdetect) it doesn’t correctly relinquish the bus until the backpack is disconnected and powered off. I’ve emailed them to see if they know why this is and how to fix it (I suspect it’s a firmware problem on the backpack, which is AVR-based… but I’m not sure how I’d update it yet).

UPDATE: the backpack works fine with the Pi – it was the nut behind the wheel that was the problem. More in an upcoming post.

My most unique purchase was the HexBright FLEX 500 lumen Arduino-powered LED flashlight! I was able to reprogram it pretty easily, though it doesn’t work with the Arduino IDE 1.5.x yet (also, when adding this target, merging their boards.txt to your boards.txt is the easy way to go… it didn’t work for me otherwise).

Note that I’m not thrilled with the charger that comes with the HexBright… I recently learned a LOT about off-brand USB chargers I was unaware of, particularly with regard to safety, from this EEVBlog post among other places, and that makes me wary of chargers that look like knockoffs. While I’m happy to use cheaper parts when possible (I’m certainly not a hardware purist!), I expect the things I plug into my wall to have a a valid UL stamp on them at the very least. Bonus points if they spell the words right on it (“inter-mation equipment”? Looks like copy-pasta from a few other questionable chargers I’ve seen teardowns of – probably the same maker slaps whatever name you want on them). And no, that serial number is not “serial” at all on cheap chargers.

Again, not knocking the flashlight, which seems pretty solid… just not thrilled with the cheap USB plug-in charger it comes with. Now that I know more about what goes into these small form-factor chargers I can appreciate that you don’t want to skimp when it comes to quality on them.

Then, in the course of updating things I updated the software for my OWON oscilloscope. Up to this point everything worked fine, but when I put the ‘scopes icon on the desktop it was the Eclipse icon, not the one that is very clearly the one for the scope.

This bugged me, having dealt with the occasional desktop icon problem before. I figured my moving Eclipse did the deed, and I was determined to fix it. I did everything: re-selected the icon in its executable, changed it to some other apps’ icon, cleared the icon cache db and rebooted (several times)… nothing changed the fact that on the desktop this icon was for Eclipse and was nothing like the icons I was selecting. Put the shortcut anywhere else and the icon looked right, but not on the desktop.

Expectation:

Reality:

At this point (actually, right from the outset on some forums) the drumbeat sound of the modern day version of “reinstall the OS” begins to echo across my search results: roll back with System Restore… roll back with System Restore! Nope… no thanks! I didn’t come this far to trash everything else I’ve done today for the sake of a stubborn icon!

(Side-note: The MVP punter’s friends, “Run System Restore”, along with the other answers of “redirect the question to the third-party software maker” and “drown the question in data requests” seem ubiquitous when it comes to semi-official channels for Windows “support”. They’re right up there with “Go away kid – you bother me!” when it comes to help. Fortunately there are far better forums out there that think outside the “it can’t be Windows – it must be something you did!” box. Unfortunately some MVP ranks appear to come free with a box of Cracker Jacks.)

Looking deeper, I noticed a peculiar “.eclipseproduct” file in the OWON folder. That made me wonder if a) this app was made with Eclipse and b) the icons I was seeing weren’t all the icons in the executable (logically, one might be an Eclipse icon for some reason, perhaps a default placeholder!) I grabbed BeCyIconGrabber (another handy and free utility) to see exactly what icons were in the exe… and the true nature of the “problem” was revealed:

Well, isn’t that special! An executable can contain numerous icons, but they are usually identical except for sizing. In this case, the 256×256 icon was defaulted to Eclipse’s (which appears to be the IDE used to develop the app).

Short of injecting a “corrected” icon into the .exe (not worth my time) it appears you would have to find a file with a valid 256×256 icon and use it for your icon source instead – or just live with the unexpected but now understandable icon, safe in the knowledge that it was neither Windows’ nor your own fault that this icon was stubbornly strange.

Funny how much better life is when you find the root cause, rather than hitting the system with the MVP’s flamethrower (bka System Restore), which would have solved exactly nothing here while adding a ton of work to re-do all the other work since that restore point was created!

And you can bet that if that didn’t work, “nuke it from orbit” (bka re-install Windows) would come next… what a great away to drive people to other OSes, even though Windows 7 is one of the most stable and robust versions of Windows out there. I’ve yet to encounter a non-virus problem in Windows that actually required doing a restore, let alone a re-install .. which has to count for something versus wearing a puffed-up acronym, parroting a troubleshooting script and racking up some sort of “points” for ineptitude in the process. (I’ve almost never found a useful answer on MS TechNet. At least on StackExchange answers are vetted by the community and points tend to mean something, while SevenForums tends to have pretty solid answers when it comes to Windows.)

tl;dr – If a particular desktop icon is stubbornly strange, particularly for a third-party app, it might be because its actually strange in the source file. If someone suggests system restore or re-installing Windows for a seemingly simple issue… look elsewhere for answers first.

In this case, an Arduino project I’ve been dabbling with for a few weeks now… I’ve successfully combined a Wii nunchuck controller and a long RGB LED strip (WS2811-based) using an Arduino Mega. It’s very much a work-in-progress, but it’s the start of building a little body of work out there.

Today I stumbled on LTspice from Linear Technology, which I found to be surprisingly intuitive (and given that it’s based on the venerable and powerful SPICE it’s that much more worth learning). It’s not as animation-friendly as Micro-Cap but it’s pretty easy to make custom components (and where else can you simulate the actual inner workings of a 555 timer or a 741 op-amp?) (And yes, the 741 is not something you’d reach for normally nowadays, but they are instructive nonetheless.)

Memristors have intrigued me since they made the news a few years back, when discoveries were made of practical memristive materials. You can read more about them on Wikipedia and so on, and though very subject is not without controversy I personally think they will prove to be a valuable technology going forward. I found them intriguing from both the EE and the MSE perspectives.

Side-note: if you find yourself wondering more about why a device does what it does (in my case, the humble MOSFET) than regurgitating Kirchoff’s Law and applying Thévenin’s theorem, you’re probably better off in Materials Science Engineering than Electrical Engineering. At least that’s what happened to me… that and a very grumpy EE prof who made it clear we’d all be obsolete, unemployed and grumpy like him soon anyway. And you can always fall back on CS when the semiconductor market tanks…

Given all that, today I decided to see if I could simulate a circuit with a memristor in it. Problem is, while some sims have memristors, LTspice does not, and apparently nobody out there has published a simple example that actually works (I found videos – who writes a text tutorial in a video? – and a broken webpage, but I never found a working sim). So, I made one… and after a few false starts I got it to work. Not bad for an evening at home!

To try my demo out, simply click here to download my LTspice-based memristor simulation and follow the instructions in the README.txt file for how to install and use it.

And to reiterate, this is a demo based on the SPICE model of a memristor as described in one paper… this is for educational purposes only and no, I can’t help you with your homework.

Note: this post applies to multiple utilities with this use case, such as NirSoft’s ChromePass and LastPass. This also applies only to Windows (though it may have similarities to workarounds for other OSes). Finally, BEWARE: this involves carefully fiddling with some folder names. There be dragons!

So, I recently decided to look at better password solutions than what the various browsers afford. I looked at KeePass, RoboForm and LastPass. Your mileage may vary, but for now I’m trying out LastPass.

In the course of this I stumbled on the fact that it’s surprisingly cumbersome to import passwords from Chrome (a browser I’ve come to like but which is at times cryptic). I ended up  turning to NirSoft’s ChromePass in an attempt to get the passwords exported from Chrome for later import. That failed, in the same way LastPass failed: it was happy to import one set of passwords only, no matter which Chrome profile was active. Note that the Windows login is the SAME in each case, thus there are multiple Chrome profiles visible at once.

Turns out, the only profile ChromePass, LastPass and others ever refer to for password data is:
C:\Users\%USERNAME%\AppData\Local\Google\Chrome\User Data\Default

If you have multiple profiles you’ve used under this Windows login, you will have folders such as:
C:\Users\%USERNAME%\AppData\Local\Google\Chrome\User Data\Default
C:\Users\%USERNAME%\AppData\Local\Google\Chrome\User Data\Profile 1
C:\Users\%USERNAME%\AppData\Local\Google\Chrome\User Data\Profile 2
etc.

However, if you’ve removed and re-added profiles you may not even HAVE a profile folder called “Default” here! (Resulting in an empty password list when you attempt to use LastPass or ChromePass to get to them.)

Thus, if your current Chrome login happens to be in that profile folder, you’ll get the info from that, but if you have another Chrome login (or for other reasons no longer have the Default profile folder) you’d be outta luck – until now.

What to do? It’s surprisingly simple! If what you see when importing or exporting passwords is NOT what you expect to see (that is, you are seeing the Default profile only), do this:

• Close Chrome (clear your cache first – always a good idea)
• Temporarily rename the profile of interest to “Default” by renaming its folder accordingly (if Default already exists, rename it to something else temporarily before doing this). You may exercise some trial and error for multiple profiles.
• Now, do what you need to do – whatever profile is in a folder called “Default” is the one that’ll open up in your utility/plugin. You can open Chrome and import passwords to LastPass, leave Chrome closed and use ChromePass to dump your passwords, etc.
• Done? Close Chrome!
• Rename the folders back to the way they were

Again, BE CAREFUL! Would that it be simpler, but at least it works. Hope you find this helpful!

PS: When importing passwords to LastPass from Firefox or Chrome do be careful: I’ve already run into cases where not all passwords get imported. It’s a bit of a pain, but it should be worth the benefit of being better organized.

UPDATE 1: While this worked for ChromePass, LastPass (as a browser extension) can be a little more tricky: after renaming the Default profile (if necessary), COPY the profile of interest to a folder named ”Default”. THEN open Chrome, do your import to LastPass, close Chrome, and remove the copy you made. 

So, I groggily log in and start looking around, and immediately see that MySQL is using LOTS of CPU all by its lonesome. I immediately wondered if somehow a hacker got in or something (given the recently reported MySQL bug) so I went to look for some hints on investigating that. On a whim (remembering the odd timing of this) I searched Google for “mysqld went to 90% cpu at midnight”. Sure enough, it’s probably a bug.

Turns out, this weekend sports a rare “leap second“… and that triggers a kernel bug which in turn causes MySQL to start eating CPU like a bad gambler at a casino buffet.

I found the following suggestion (found here and here) worked well enough for me on its own (fingers crossed anyway):

sudo date -s "`date`"  # <-- Mind the backticks!

I ran that and the problem immediately disappeared. I didn’t need to reboot or restart mysqld (rebooting would also fix it apparently, but restarting mysqld would not).

Hopefully this bit of advice serves you as well!

Failed to parse file C:\Eclipse_workspace\ButtonTest2\res\drawable\arrow_1.xml
Exception details are logged in Window > Show View > Error Log

The error itself isn’t exactly informative, and certainly doesn’t make sense:

org.xmlpull.v1.XmlPullParserException: PI must not start with xml (position:unknown @1:5 in java.io.InputStreamReader@1c167686)
at org.kxml2.io.KXmlParser.exception(Unknown Source)
…….

Hmm. XML files should start with an appropriate declaration line, yet this says my XML file shouldn’t have it. That’s… weird. Normally I just remove the line til it stops complaining but as I’m playing with the second pre-release of r17 android SDK (some key components have been broken lately) and feeling bold I decided to understand this once and for all. What follows is the text of my answer on StackOverflow earlier tonight:

I’ve beaten my head against this wall a few times and now I have the answer (I *hate* mysterious behavior, especially when it seems to be random).

I just ran a 4-way test (Unix and DOS, with and without the binary “EF BB BF” BOM header/signature). I found that Eclipse does not put a binary BOM header into XML files it creates, period. If you add a file that has the `”<?xml … ?>”` declaration header, it works correctly ONLY if that binary signature is ABSENT. (Whether EOLs are Unix or DOS doesn’t matter). If you have *both* forms of header you get this problem. You can have one, the other, or neither, but never both.

So, if you create a new XML file in Eclipse you can expect (in 3.6 – 3.7.1 anyway) that it has no binary BOM header and you should properly put a `”<?xml … ?>”` declaration header in there to identify it as Unicode. If you IMPORT a project that has files with binary BOM header signatures or use an external editor that likes to add them, that’s where all this fun begins.

Ideally, ensure XML files with declaration don’t have binary BOM header before importing them… otherwise, remove that `”<?xml … ?>”` declaration header whenever Eclipse complains… short of removing the binary header that’s all you can readily do. (It’d be nice if Eclipse handled this more consistently… or at least gave you a way to identify and fix the problem readily by removing the binary BOM header…)

* If in doubt, examine the file with an external hex editor, preferably one that is NOT Unicode-aware (WinHex worked for me, YMMV). Some text/hex editors will auto-detect Unicode and may auto-add a header when the file didn’t really have one.

Postscript: I use IDM’s UltraEdit / UltraCompare primarily. They work great but they’re a little too good at determining if a file is Unicode or not, to the point where these four text files ALL had the binary BOM header in hex mode!! WinHex showed the exact situation, and indeed only two of the files actually had that header. However, I’d like my regular editor to work correctly. Time to write IDM an email…

Unlike a desktop app, where one could add a “safe mode” icon for such instances, there are few options in the mobile space. These include:

• Letting the user delete the application data manually via the Settings menu
• Automatically resetting to defaults after a crash
• Attempting to automatically work around the broken area
• Asking the user on restart if they want to reset to defaults
• Falling back to the last known-good configuration

It’s one thing to be a power user, but another to be forced to become one thus the first option isn’t really practical. The second option guarantees recovery but may degrade the user experience (especially if there are lots of settings involved)… it should be a last resort, and a user choice. The third option is nice, but can become quite complex: this works best if settings are checkpointed frequently, but such frequent saves to flash memory are not usually a great idea. In addition, the more complex the fail-safe, the more likely it may cause a triggering exception.

Asking the user first is an important part of any solution, but it requires architecting a fail-safe startup sequence (one that is preference invariant). One can then offer the choice of a “fresh start” or the option of using a backed-up configuration, ensuring that the user has at least a chance of restoring to a previously working state with their settings intact.

A useful pattern for a single-instance app would be the following, using a persistent IsRunning flag:

• App launches
• Load settings
• Check if the IsRunning flag is set: if it is, then the last run exited abnormally (none of the normal exit points were hit which would’ve cleared the flag):
• Clear the IsRunning flag
• Offer to restore from backup / defaults or attempt to continue as-is
• Initialize states and display initial UI presentation
• Save the current settings with a backup name
• Set the IsRunning flag
• Save the settings normally
• On normal app exit / backgrounding, clear the IsRunning flag and save settings normally

There are other things you can do, including instrumenting for crash data collection and presenting the user an option to forward that data (anonymized!) to you for analysis.

Software quality is as much about how well you avoid defects in the first place as it is about how gracefully you recover from them!^† Mistakes happen, especially in apps with complex GUIs. Errors should be as rare as possible, but when errors occur it’s the unrecoverable ones that lead to lost users. Solid error handling and recovery goes a long way towards satisfied users who stick around for the release that irons out the bug they encountered and worked around

^†This is a critical difference between hardware and software quality, especially for smaller shops. If you ship a physically defective device, the typical recourse is a return for repair or replacement – a costly proposition in terms of logistics as well as reworked hardware / waste. The quality patterns for software and hardware have commonalities but are generally quite different in practice: in general, hardware is by its very nature far more costly and less forgiving of quality failures. This is why popular hardware quality initiatives tend to translate poorly to the software space.

Consider apps that are meant to be full-screen, and may be deeply interactive (such as a game). Having the action bar onscreen is not tenable, so one hides it. Whereas before you could use the menu button in the gutter (where “home” and “back” reside and indeed where “menu” appears – only for legacy apps), now you must either code your own button or invent a unique (but intuitive!) gesture to accomplish the end result of opening a custom menu or showing the action bar. If that’s not UI fragmentation I don’t know what is!

Though I find the action bar awkwardly placed and non-intuitive for in-app interactions, I understand the merit of having a unified UI. What I find fault with is the shortsighted decision to effectively remove an important hard button from Android (does anyone miss the “search” button?)

Now we have only two guaranteed (not hideable) buttons left, for the purpose of app navigation: “back” and “home” – along with a third new button for fast task switching (which is a little redundant… long-press on the home button could easily have the same effect without adding buttons). In addition (and likely for the foreseeable future) we have a “menu” button that only appears for legacy apps (thus that space in the toolbar is already “reserved” and will remain so for quite some time).

It’s too bad… there’s a strong argument for always having a “menu” button handy in an always-visible place. Maybe it unhides your action bar, or maybe it opens a custom menu, but removing it doesn’t help matters: it simply encourages a new kind of fragmentation for apps that don’t need an action bar onscreen normally. Now, instead of having a consistent way to perform that “open menu” action everyone gets to do it their own way (or leave the action bar open full-time, which significantly detracts from some apps).

One of many things that I like about Android is that the basic controls (home, menu, back) are intuitive and reasonable, balancing function with minimalism. Contrast this with the one overloaded hard button in another popular OS: much like a one-button mouse, oversimplification leads to confusion, reduced functionality and a fragmented UI as everyone works around the limitation differently according to their app’s design.

The rationale for taking away the menu button – one of the three main “always-visible” navigation and control buttons for Android up to now – defies logic: it fails at reducing fragmentation and funnels developers (and thus users) into an awkward, one-size-fits-all paradigm that is the antithesis of Android. This loss of a “hard” menu button is an unwelcome reduction in basic functionality in an apparent nod to the oversimplification of other OSes. I don’t find fault with the existence of the action bar, but removing the dedicated menu button from the always-visible area of the screen was IMHO extremely shortsighted. Now, instead of having a convenient way to reveal a hidden action bar, one must figure out how to do this for each app that normally hides its action bar. All this for the sake of removing a button that appears anyway for legacy apps.

As part of this, I needed to rebuild the emulator. It was not as straightforward as I’d hoped so I decided to write up a little howto here in case others decide to try rebuilding it on Windows. It’s really not hard at all once you install a few prerequisites! [Note: You'll also find info here about building this via Linux.]

[Edit: As of this post the HEAD is 6e2f62...599648 for platform_external_qemu. While this post-r12 work WILL build in Windows with these procedures, I just found out that the older r12 code (tagged as android-sdk-tools_r12) will NOT build in Windows! You'd need to do r12-based builds in Linux with MinGW installed. It's interesting to see just how much was fixed since SDK r12... including, evidently, the very issue I discovered today! Hopefully SDK r13 gets released soon. Oh look, it's out! Interestingly, unlike r12 there is no corresponding r13 tag for this one... which is part of why I didn't notice it in the Git log (I saw the tools_r13 branch though there are lots of changes after that).]

Note: If you choose other install dirs, adjust the values below accordingly!

Install Python 2.6 or 2.7:

• I used ActivePython 2.7.2.5 x64 (x86 should be equally good)
• Download: http://www.activestate.com/activepython/downloads
• Default bin dir for Python = C:\Python27

Install MinGW (it includes msys now):

• Information: http://www.mingw.org/wiki/MinGW
• Download: http://sourceforge.net/projects/mingw/
• Default bin dir for MinGW = C:\MinGW\bin
• Default bin dir for msys = C:\MinGW\msys\1.0\bin

Install Git: I use msysgit because I also use (and happily recommend!) TortoiseGit 1.7.3.0:

• Information: http://code.google.com/p/msysgit/
• Download: http://code.google.com/p/msysgit/downloads/list
• Git-1.7.4-preview20110204.exe works, 1.7.6 may as well
• Default bin dir for Git = C:\Git\bin

Open a command prompt and ensure your path is set correctly (omit duplicates as necessary, but ensure such entries weren’t eclipsed by other things you’ve installed, like Cygwin):

set PATH=C:\MinGW\msys\1.0\bin;C:\MinGW\bin;C:\Python27;C:\Git\bin;%PATH%

Clone the repository! (Because of the kernel.org outage we’re using the github mirror):

mkdir H:\temp_repo
cd /D H:\temp_repo
git clone https://github.com/android/platform_external_qemu.git

Start bash in the folder with the emulator sources (from this point on commands run at the bash prompt):

cd /D H:\temp_repo\platform_external_qemu
bash

Setting PS1 makes the prompt more verbose than the default “bash-3.1$” prompt:

PS1='\[\e]0;\w\a\]\n\[\e[36m\]\u@\h \[\e[32m\]\w\[\e[0m\]\n\$ '

There’s a small problem if you want the version string in the emulator to be fully formed:

• In Makefile.common, ANDROID_BUILD_ID get set only if BUILD_STANDALONE_EMULATOR == “” – never if BUILD_STANDALONE_EMULATOR == ‘true’ (which is what we’re building here). Here’s a fix for that, at least for the 13.x line of the emulator (line numbers may vary!):
• Broken: line 66: ifeq ($(BUILD_STANDALONE_EMULATOR),)
• Patched: line 66: ifneq ($(BUILD_STANDALONE_EMULATOR),false)
• If in doubt, ignore this tweak! Understand your -version output will not be very informative. That can quickly get annoying if you are building multiple versions…

Set some variables and run the build:

# Customize this as desired
export ANDROID_SDK_TOOLS_REVISION=42
export ANDROID_BUILD_ID=custom

# Inserts your username and the timestamp in the build name
export MY_BUILD_TIMESTAMP=`date +%Y%m%d_%H%M%S`
export BUILD_ID=${USERNAME}
export BUILD_NUMBER=${MY_BUILD_TIMESTAMP}

# Executes the build
./android-rebuild.sh --static 2>&1 | \
   tee build_${USERNAME}-${MY_BUILD_TIMESTAMP}.log

Check the version string of the emulator you just built:

• Run this:
• objs/emulator.exe -version
• See output like this:
• Android emulator version 13.0 (build_id Marty-20110925_042430)

Using the newly-built emulator:

• The objects of interest in objs are:
• emulator.exe, emulator-arm.exe, emulator-x86.exe
• You may need to copy them to android-sdk-windows/tools folder to use them (back up the originals first before replacing them!)

Notes on building via Linux:

As mentioned above, I managed to get all this to work on Windows just fine… technically, no source edits are necessary at this point to make it work. However, when I switched to the android-sdk-tools_r12 tag it failed with:

cc1: error: unrecognized command line option “-mno-cygwin”

Turns out this is a known issue… and in fact it was fixed in a later commit to the emulator code base! (I saw it as I was browsing the overall delta between r12 and the latest.) But, even with that fixed I couldn’t get r12 to build within Windows, so I went to Linux. I was already configured to build CyanogenMod, so everything was in place except MinGW. I installed mingw32, made that one fix to android-rebuild.sh and ran:

./android-rebuild.sh –static –mingw

And it worked just fine!

(Note: the –mingw option is NOT used when building this within Windows… it’s a little confusing because of course we’re using MinGW in Windows, but that’s how it’s coded in their build system.)

This is all nice, but when you enable hybrid sleep you lose “hibernate” as a separate option. Perhaps this is to avoid confusion (remember: hybrid sleep performs hibernation as part of its process, it just doesn’t completely power down). However, there are times I’d rather have the system fully hibernate instead of just sleeping (such as if I’m going to be away a few days). Since touching the power button once it’s in full hybrid sleep often re-awakens my system (and I don’t prefer to power down directly from any sleep state), I wanted a one-click solution.

So, if you use hybrid sleep and want to explicitly hibernate, run this (even as a regular user) and your system should hibernate rather than just enter hybrid sleep:

%windir%\system32\rundll32.exe PowrProf.dll, SetSuspendState 0,1,0

This should also work if you DON’T use hybrid sleep and DO have the hibernation option… though if you have “hibernate” as an option you’re probably not looking for this sort of fix.

Disclaimers: If hibernation or hybrid sleep didn’t work before this won’t make hibernation work (e.g., if you have insufficient disk space or have explicitly disabled hibernation and hybrid sleep)… in that case this will likely send you into the typical sleep mode. You can usually tell the difference by noting the lengthy time spend writing data to the disk, followed by a clear power off. (Hybrid sleep does the same, except it goes into sleep mode – how your computer shows this will vary).

More info here.

2011-09-12 update: The version here is for Python 3.x as described below. Jens Diemer posted an updated version of this to GitHub with corrections. It’s mainly a backport of this script to Python 2.x along with some cosmetic changes (and yes, I do like delimiting functions and such with comment lines – it makes code blocks far easier to distinguish, even in indentation-intensive Python). He added a bugfix to handle SIGBREAK better, which I’ve pulled into the code below.

A while back, when I decided to learn Python, I looked into writing my own sort of ping utility with it. Then I learned just how ridiculously arduous it is to perform this seemingly-simple function (essentially, you have to be running as root in Unix (or Administrator in Windows) to accomplish this, and generally speaking running scripts like this as root is considered bad form). But it can be done…

This intrigued me as a pet project but I moved on to other things, eventually picking up Python along the way (I’ve focused on Python 3.x since the Unicode handling is FAR better than it is in 2.7, not to mention its other improvements).

Recently, I decided to revisit the “ping with Python” idea, and searched around to see if anyone else had tackled it. Sure enough, I found George Notaras’ site which has a very nice implementation in 2.x as well as a recent commenter who created a 3.x port. Only problem was, the HTML is mangled there so neither script worked. As I delved into it, I managed to get the scripts working, and then decided to go the extra mile and make some enhancements and improvements. Along the way I ended up getting the source for ping (via Cygwin) and building it from scratch as well (particularly to understand the checksum routine, and for other hints in general).

It was rather pleasing to not only get this working but to make it work far better in terms of functionality and robustness, all the while reinforcing and extending my understanding of Python.

Note: The original C source for “ping” can be found here (you’ll need to extract it from ping.shar). Or just search for ping.c… for example, I found an identical copy of ping.c here.

#!/usr/bin/env python
# -*- coding: utf-8 -*-
 
"""
    A pure python ping implementation using raw sockets.
 
    (This is Python 3 port of https://github.com/jedie/python-ping)
 
    Note that ICMP messages can only be sent from processes running as root
    (in Windows, you must run this script as 'Administrator').
 
    Derived from ping.c distributed in Linux's netkit. That code is
    copyright (c) 1989 by The Regents of the University of California.
    That code is in turn derived from code written by Mike Muuss of the
    US Army Ballistic Research Laboratory in December, 1983 and
    placed in the public domain. They have my thanks.
 
    Bugs are naturally mine. I'd be glad to hear about them. There are
    certainly word - size dependencies here.
 
    Copyright (c) Matthew Dixon Cowles, <http://www.visi.com/~mdc/>.
    Distributable under the terms of the GNU General Public License
    version 2. Provided with no warranties of any sort.
 
    Original Version from Matthew Dixon Cowles:
      -> ftp://ftp.visi.com/users/mdc/ping.py
 
    Rewrite by Jens Diemer:
      -> http://www.python-forum.de/post-69122.html#69122
 
    Rewrite by George Notaras:
      -> http://www.g-loaded.eu/2009/10/30/python-ping/
 
    Enhancements by Martin Falatic:
      -> http://www.falatic.com/index.php/39/pinging-with-python
 
    Revision history
    ~~~~~~~~~~~~~~~~
 
    October 12, 2011
    --------------
    Merged updates from the main project
      -> https://github.com/jedie/python-ping
 
    September 12, 2011
    --------------
    Bugfixes + cleanup by Jens Diemer
    Tested with Ubuntu + Windows 7
 
    September 6, 2011
    --------------
    Cleanup by Martin Falatic. Restored lost comments and docs. Improved
    functionality: constant time between pings, internal times consistently
    use milliseconds. Clarified annotations (e.g., in the checksum routine).
    Using unsigned data in IP & ICMP header pack/unpack unless otherwise
    necessary. Signal handling. Ping-style output formatting and stats.
 
    August 3, 2011
    --------------
    Ported to py3k by Zach Ware. Mostly done by 2to3; also minor changes to
    deal with bytes vs. string changes (no more ord() in checksum() because
    >source_string< is actually bytes, added .encode() to data in
    send_one_ping()).  That's about it.
 
    March 11, 2010
    --------------
    changes by Samuel Stauffer:
    - replaced time.clock with default_timer which is set to
      time.clock on windows and time.time on other systems.
 
    November 8, 2009
    ----------------
    Improved compatibility with GNU/Linux systems.
 
    Fixes by:
     * George Notaras -- http://www.g-loaded.eu
    Reported by:
     * Chris Hallman -- http://cdhallman.blogspot.com
 
    Changes in this release:
     - Re-use time.time() instead of time.clock(). The 2007 implementation
       worked only under Microsoft Windows. Failed on GNU/Linux.
       time.clock() behaves differently under the two OSes[1].
 
    [1] http://docs.python.org/library/time.html#time.clock
 
    May 30, 2007
    ------------
    little rewrite by Jens Diemer:
     -  change socket asterisk import to a normal import
     -  replace time.time() with time.clock()
     -  delete "return None" (or change to "return" only)
     -  in checksum() rename "str" to "source_string"
 
    December 4, 2000
    ----------------
    Changed the struct.pack() calls to pack the checksum and ID as
    unsigned. My thanks to Jerome Poincheval for the fix.
 
    November 22, 1997
    -----------------
    Initial hack. Doesn't do much, but rather than try to guess
    what features I (or others) will want in the future, I've only
    put in what I need now.
 
    December 16, 1997
    -----------------
    For some reason, the checksum bytes are in the wrong order when
    this is run under Solaris 2.X for SPARC but it works right under
    Linux x86. Since I don't know just what's wrong, I'll swap the
    bytes always and then do an htons().
 
    ===========================================================================
    IP header info from RFC791
      -> http://tools.ietf.org/html/rfc791)
 
    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |Version|  IHL  |Type of Service|          Total Length         |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |         Identification        |Flags|      Fragment Offset    |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |  Time to Live |    Protocol   |         Header Checksum       |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                       Source Address                          |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                    Destination Address                        |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                    Options                    |    Padding    |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 
    ===========================================================================
    ICMP Echo / Echo Reply Message header info from RFC792
      -> http://tools.ietf.org/html/rfc792
 
        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        |     Type      |     Code      |          Checksum             |
        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        |           Identifier          |        Sequence Number        |
        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        |     Data ...
        +-+-+-+-+-
 
    ===========================================================================
    ICMP parameter info:
      -> http://www.iana.org/assignments/icmp-parameters/icmp-parameters.xml
 
    ===========================================================================
    An example of ping's typical output:
 
    PING heise.de (193.99.144.80): 56 data bytes
    64 bytes from 193.99.144.80: icmp_seq=0 ttl=240 time=127 ms
    64 bytes from 193.99.144.80: icmp_seq=1 ttl=240 time=127 ms
    64 bytes from 193.99.144.80: icmp_seq=2 ttl=240 time=126 ms
    64 bytes from 193.99.144.80: icmp_seq=3 ttl=240 time=126 ms
    64 bytes from 193.99.144.80: icmp_seq=4 ttl=240 time=127 ms
 
    ----heise.de PING Statistics----
    5 packets transmitted, 5 packets received, 0.0% packet loss
    round-trip (ms)  min/avg/max/med = 126/127/127/127
 
    ===========================================================================
"""
 
#=============================================================================#
import os, sys, socket, struct, select, time, signal
 
if sys.platform == "win32":
    # On Windows, the best timer is time.clock()
    default_timer = time.clock
else:
    # On most other platforms the best timer is time.time()
    default_timer = time.time
 
#=============================================================================#
# ICMP parameters
 
ICMP_ECHOREPLY  =    0 # Echo reply (per RFC792)
ICMP_ECHO       =    8 # Echo request (per RFC792)
ICMP_MAX_RECV   = 2048 # Max size of incoming buffer
 
MAX_SLEEP = 1000
 
class MyStats:
    thisIP   = "0.0.0.0"
    pktsSent = 0
    pktsRcvd = 0
    minTime  = 999999999
    maxTime  = 0
    totTime  = 0
    fracLoss = 1.0
 
myStats = MyStats # Used globally
 
#=============================================================================#
def checksum(source_string):
    """
    A port of the functionality of in_cksum() from ping.c
    Ideally this would act on the string as a series of 16-bit ints (host
    packed), but this works.
    Network data is big-endian, hosts are typically little-endian
    """
    countTo = (int(len(source_string)/2))*2
    sum = 0
    count = 0
 
    # Handle bytes in pairs (decoding as short ints)
    loByte = 0
    hiByte = 0
    while count < countTo:
        if (sys.byteorder == "little"):
            loByte = source_string[count]
            hiByte = source_string[count + 1]
        else:
            loByte = source_string[count + 1]
            hiByte = source_string[count]
        sum = sum + (hiByte * 256 + loByte)
        count += 2
 
    # Handle last byte if applicable (odd-number of bytes)
    # Endianness should be irrelevant in this case
    if countTo < len(source_string): # Check for odd length
        loByte = source_string[len(source_string)-1]
        sum += loByte
 
    sum &= 0xffffffff # Truncate sum to 32 bits (a variance from ping.c, which
                      # uses signed ints, but overflow is unlikely in ping)
 
    sum = (sum >> 16) + (sum & 0xffff)    # Add high 16 bits to low 16 bits
    sum += (sum >> 16)                    # Add carry from above (if any)
    answer = ~sum & 0xffff              # Invert and truncate to 16 bits
    answer = socket.htons(answer)
 
    return answer
 
#=============================================================================#
def do_one(destIP, timeout, mySeqNumber, numDataBytes):
    """
    Returns either the delay (in ms) or None on timeout.
    """
    global myStats
 
    delay = None
 
    try: # One could use UDP here, but it's obscure
        mySocket = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.getprotobyname("icmp"))
    except socket.error as e:
        print("failed. (socket error: '%s')" % e.args[1])
        raise # raise the original error
 
    my_ID = os.getpid() & 0xFFFF
 
    sentTime = send_one_ping(mySocket, destIP, my_ID, mySeqNumber, numDataBytes)
    if sentTime == None:
        mySocket.close()
        return delay
 
    myStats.pktsSent += 1;
 
    recvTime, dataSize, iphSrcIP, icmpSeqNumber, iphTTL = receive_one_ping(mySocket, my_ID, timeout)
 
    mySocket.close()
 
    if recvTime:
        delay = (recvTime-sentTime)*1000
        print("%d bytes from %s: icmp_seq=%d ttl=%d time=%d ms" % (
            dataSize, socket.inet_ntoa(struct.pack("!I", iphSrcIP)), icmpSeqNumber, iphTTL, delay)
        )
        myStats.pktsRcvd += 1;
        myStats.totTime += delay
        if myStats.minTime > delay:
            myStats.minTime = delay
        if myStats.maxTime < delay:
            myStats.maxTime = delay
    else:
        delay = None
        print("Request timed out.")
 
    return delay
 
#=============================================================================#
def send_one_ping(mySocket, destIP, myID, mySeqNumber, numDataBytes):
    """
    Send one ping to the given >destIP<.
    """
    destIP  =  socket.gethostbyname(destIP)
 
    # Header is type (8), code (8), checksum (16), id (16), sequence (16)
    myChecksum = 0
 
    # Make a dummy heder with a 0 checksum.
    header = struct.pack(
        "!BBHHH", ICMP_ECHO, 0, myChecksum, myID, mySeqNumber
    )
 
    padBytes = []
    startVal = 0x42
    for i in range(startVal, startVal + (numDataBytes)):
        padBytes += [(i & 0xff)]  # Keep chars in the 0-255 range
    data = bytes(padBytes)
 
    # Calculate the checksum on the data and the dummy header.
    myChecksum = checksum(header + data) # Checksum is in network order
 
    # Now that we have the right checksum, we put that in. It's just easier
    # to make up a new header than to stuff it into the dummy.
    header = struct.pack(
        "!BBHHH", ICMP_ECHO, 0, myChecksum, myID, mySeqNumber
    )
 
    packet = header + data
 
    sendTime = time.time()
 
    try:
        mySocket.sendto(packet, (destIP, 1)) # Port number is irrelevant for ICMP
    except socket.error as e:
        print("General failure (%s)" % (e.args[1]))
        return
 
    return sendTime
 
#=============================================================================#
def receive_one_ping(mySocket, myID, timeout):
    """
    Receive the ping from the socket. Timeout = in ms
    """
    timeLeft = timeout/1000
 
    while True: # Loop while waiting for packet or timeout
        startedSelect = time.time()
        whatReady = select.select([mySocket], [], [], timeLeft)
        howLongInSelect = (time.time() - startedSelect)
        if whatReady[0] == []: # Timeout
            return None, 0, 0, 0, 0
 
        timeReceived = time.time()
 
        recPacket, addr = mySocket.recvfrom(ICMP_MAX_RECV)
 
        ipHeader = recPacket[:20]
        iphVersion, iphTypeOfSvc, iphLength, \
        iphID, iphFlags, iphTTL, iphProtocol, \
        iphChecksum, iphSrcIP, iphDestIP = struct.unpack(
            "!BBHHHBBHII", ipHeader
        )
 
        icmpHeader = recPacket[20:28]
        icmpType, icmpCode, icmpChecksum, \
        icmpPacketID, icmpSeqNumber = struct.unpack(
            "!BBHHH", icmpHeader
        )
 
        if icmpPacketID == myID: # Our packet
            dataSize = len(recPacket) - 28
            return timeReceived, dataSize, iphSrcIP, icmpSeqNumber, iphTTL
 
        timeLeft = timeLeft - howLongInSelect
        if timeLeft <= 0:
            return None, 0, 0, 0, 0
 
#=============================================================================#
def dump_stats():
    """
    Show stats when pings are done
    """
    global myStats
 
    print("\n----%s PYTHON PING Statistics----" % (myStats.thisIP))
 
    if myStats.pktsSent > 0:
        myStats.fracLoss = (myStats.pktsSent - myStats.pktsRcvd)/myStats.pktsSent
 
    print("%d packets transmitted, %d packets received, %0.1f%% packet loss" % (
        myStats.pktsSent, myStats.pktsRcvd, 100.0 * myStats.fracLoss
    ))
 
    if myStats.pktsRcvd > 0:
        print("round-trip (ms)  min/avg/max = %d/%0.1f/%d" % (
            myStats.minTime, myStats.totTime/myStats.pktsRcvd, myStats.maxTime
        ))
 
    print()
    return
 
#=============================================================================#
def signal_handler(signum, frame):
    """
    Handle exit via signals
    """
    dump_stats()
    print("\n(Terminated with signal %d)\n" % (signum))
    sys.exit(0)
 
#=============================================================================#
def verbose_ping(hostname, timeout = 1000, count = 3, numDataBytes = 55):
    """
    Send >count< ping to >destIP< with the given >timeout< and display
    the result.
    """
    global myStats
 
    signal.signal(signal.SIGINT, signal_handler)   # Handle Ctrl-C
    if hasattr(signal, "SIGBREAK"):
        # Handle Ctrl-Break e.g. under Windows
        signal.signal(signal.SIGBREAK, signal_handler)
 
    myStats = MyStats() # Reset the stats
 
    mySeqNumber = 0 # Starting value
 
    try:
        destIP = socket.gethostbyname(hostname)
        print("\nPYTHON PING %s (%s): %d data bytes" % (hostname, destIP, numDataBytes))
    except socket.gaierror as e:
        print("\nPYTHON PING: Unknown host: %s (%s)" % (hostname, e.args[1]))
        print()
        return
 
    myStats.thisIP = destIP
 
    for i in range(count):
        delay = do_one(destIP, timeout, mySeqNumber, numDataBytes)
 
        if delay == None:
            delay = 0
 
        mySeqNumber += 1
 
        # Pause for the remainder of the MAX_SLEEP period (if applicable)
        if (MAX_SLEEP > delay):
            time.sleep((MAX_SLEEP - delay)/1000)
 
    dump_stats()
 
#=============================================================================#
if __name__ == '__main__':
 
    # These should work:
    verbose_ping("heise.de")
    verbose_ping("google.com")
 
    # Inconsistent on Windows w/ ActivePython (Python 3.2 resolves correctly
    # to the local host, but 2.7 tries to resolve to the local *gateway*)
    verbose_ping("localhost")
 
    # Should fail with 'getaddrinfo failed':
    verbose_ping("foobar_url.foobar")
 
    # Should fail (timeout), but it depends on the local network:
    verbose_ping("192.168.255.254")
 
    # Should fails with 'The requested address is not valid in its context':
    verbose_ping("0.0.0.0")
 
#=============================================================================#

Recently a friend on Twitter sent a link as part of a message:

“Look! it’s you in this picture.. [along with a tinyurl link]”

Other variants of this exist, such as “you’ll laugh when you see this pic of you… [along with a tinyurl link]“.

Turns out, my friend’s account had been hijacked. (Yes, it really was a friend… I’ve seen this before with Twitter and other sites but it looks identical to an attack I saw about a month ago).

In this case, the bogus link went to a subpage on itwittiler.com (IP address 220.164.140.252), a domain registered in China - earlier today! The page looks like Twitter’s login page but it’s not. In fact, I sandboxed it and entered a bogus username/password to see if it’d redirect to Twitter no matter what was entered. No… it redirected me to a “StalkTrak” page, clearly malformed and totally bogus. Read this Sophos security article for more info.

Another subpage (found via some searching online) does redirect to Twitter, but to a user who doesn’t exist. In all cases, the first page you go to looks just like the Twitter login page (there are some subtle yet sloppy differences, but next time there might not be.)

It’s quite likely the page is harvesting credentials along the way. If you get stung, go to the real Twitter homepage and reset your password pronto (after closing your browser windows).

More info on itwittiler.com, via whois:

Domain Name: ITWITTILER.COM
Registrar: JIANGSU BANGNING SCIENCE & TECHNOLOGY CO. LTD
Whois Server: whois.55hl.com
Referral URL: http://www.55hl.com
Name Server: DNS5.4CUN.COM
Name Server: DNS6.4CUN.COM
Status: ok
Updated Date: 13-aug-2011
Creation Date: 13-aug-2011
Expiration Date: 13-aug-2012

Update: Another StalkTrak spam domain made the rounds last month: itiwitter.com. Surprise! It’s registered to the very same registrar! Judging by the same subpage structure I strongly suspect it’s the very same scammer using the very same site (or site structure)… while the IP address won’t resolve, it turns out this site’s for sale! And the IP address when last they checked? 220.164.140.252. Isn’t that special?

Domain Name: ITIWITTER.COM
Registrar: JIANGSU BANGNING SCIENCE & TECHNOLOGY CO. LTD
Whois Server: whois.55hl.com
Referral URL: http://www.55hl.com
Name Server: DNS5.4CUN.COM
Name Server: DNS6.4CUN.COM
Status: clientHold
Status: clientTransferProhibited
Updated Date: 20-jul-2011
Creation Date: 15-jul-2011
Expiration Date: 15-jul-2012

Still more information can be found on this site, which talks about the same scam (back before the scammers changed domain names and came back from the “dead”).

For web/email hosting, I was with Concentric way back when (later bought by XO.com). I decided to leave XO.com when it was clearly unresponsive to the fact that the shared SMTP address was starting to routinely get blacklisted because spammers were hosted on the same machine my sites were unfortunately on. Instead of dealing with the spammers the situation was left as-is and was untenable.

I looked around and found Hosting Matters was recommended by people at the time. You can read about them elsewhere, for better or worse. They tended to be pretty responsive but I got tired of the service interruptions on the hosts I was on. Perhaps I was just unlucky that every server I was on ended up overloaded or DDoSed, but after moving servers two or three times it got old.

I also learned much later that Hosting Matters is also well-known for hosting right-wing / highly conservative websites (apparently this is why they were getting attacked frequently and why they were pretty good at fending off such attacks). This wasn’t a good fit for me either – I prefer my host to be if anything centrist/neutral, not a lightning rod.

After the latest little outage I decided to bite the bullet and migrate to HostGator. It’s also CPanel-based and has most of the same features I was used to (I’m happy to be able to custom configure SpamAssassin again, and the resource-hogging mailscanner is absent which may also be good). Otherwise the features are almost all identical or slightly better (apparently Hosting Matters had automatic backups I was unaware of, which I’m not seeing on HostGator, but that also might be why my Hosting Matters server kept bogging down late at night).

I’m still getting used to it but already there’s at least two clear differences: the server seems remarkably faster (both in general bandwidth as well as in low load) and I was able to update WordPress and its plugins directly from the WordPress admin panels (something I was never able to do on the previous host). Considering I simply exported and imported the databases and then created/extracted the tarball of the full sites (including the WordPress installations) I was totally shocked that it just worked (I gave it a shot thinking it’d fail and I’d investigate what might be wrong…) The relevant permissions and groups seem the same though.

Note: I didn’t use HostGator’s migration service… instead, I configured and migrated it all by hand (for the experience).

The HostGator bottom line: pleasantly surprised and satisfied so far!

For VPS, I was with Slicehost for over two years with a standard single slice. It was interesting and I liked the interface, but their recent acquisition by Rackspace was followed by an announcement that Rackspace will discontinue the Slicehost VPS offering in favor of their Cloud offering. I may go back to them later just to try out the whole cloud computing paradigm, but that isn’t really suitable for my VPS needs.

Seeing a lot of people migrating to Linode, I looked into it. For the same money I get more space and memory as well as the ability to use StackScripts, which should make deploying or rebuilding a server as easy as clicking a button.

I used my tried-and-true cheat sheet (which I previously used with my Slicehost slice) to create a new Ubuntu 11.04 x64 linode. I had to make a couple of minor changes (I was previously on Ubuntu 10.10) but otherwise my new Linode worked flawlessly!

The Linode bottom line: satisfied with the better value and knowing that it’s not disappearing anytime soon.

** I’ve pondered moving all my hosting to that VPS, but it doesn’t make sense, particularly for the email part. (I’m not thrilled with the idea of outsourcing my email to a third party either.) It’s a ton of maintenance and upkeep which shared hosting avoids. Tradeoff is potential partial or full outages, but hopefully those stay rare.

First, the tail of the installation output for the samba4 package:

…
module samba_dsdb initialization failed : No such object
Unable to load modules for /var/lib/samba/private/sam.ldb: dsdb_module_search_dn: did not find base dn @ROOTDSE (0 results)
Traceback (most recent call last):
File “/usr/share/samba/setup/upgradeprovision”, line 1597, in <module>
ldbs = get_ldbs(paths, creds, session, lp)
File “/usr/lib/python2.7/dist-packages/samba/upgradehelpers.py”, line 159, in get_ldbs
ldbs.sam = SamDB(paths.samdb, session_info=session, credentials=creds, lp=lp, options=["modules:samba_dsdb"])
File “/usr/lib/python2.7/dist-packages/samba/samdb.py”, line 53, in __init__
options=options)
File “/usr/lib/python2.7/dist-packages/samba/__init__.py”, line 110, in __init__
self.connect(url, flags, options)
File “/usr/lib/python2.7/dist-packages/samba/samdb.py”, line 66, in connect
options=options)
_ldb.LdbError: (80, ‘dsdb_module_search_dn: did not find base dn @ROOTDSE (0 results)’)
dpkg: error processing samba4 (–configure):
subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
samba4
E: Sub-process /usr/bin/dpkg returned an error code (1)

I first looked around… a few people had reported similar issues but the answers weren’t useful. The most common one in several places was, “Why are you using samba 4? It’s experimental!” I’ve heard that only samba4 works (well | at all?) with Windows 7 protocols. Not sure about that but it was clear that downgrading to regular samba (v3) wasn’t an ideal solution. (Note: it’s been a while since I dealt with interop issues on samba… I apparently originally installed samba4 in 10.04 or 10.10 as part of trying to talk with Windows 7.)

Finally, having done due diligence, I posted my problem as a bug report. After a few iterations it was mentioned that I should “purge” the samba database. Hmm. I didn’t see that mentioned anywhere else when I searched before (on my particular install error). So I tried it and lo and behold! The purge worked!

So, to install/upgrade samba4 on Ubuntu 11.04, you can try the following (Disclaimer: always back up first!):

# Attempt to install samba4:

sudo apt-get install samba4

# Assuming it fails to install correctly
#("_ldb.LdbError: (80, 'dsdb_module_search_dn: did not find
# base dn @ROOTDSE (0 results)')" is the error I was dealing
# with), purge samba4:

sudo apt-get purge samba4

# Finally, attempt to install samba4 again:

sudo apt-get install samba4

That should do it. If it doesn’t do it, maybe you are having a different issue.

EDIT: There’s some useful info on this page about configuring Ubuntu 11.04 Samba so as to play nice with Windows. Also, your smb.conf is probably in /etc/samba… installing samba doesn’t install a default config file even if this one is absent. I’m not sure why yet but I’d like to know since my existing one is throwing warnings when you run “samba” as a command (several “unknown parameters” are getting “ignored”… probably fine but I wish I had a clean smb.conf for this release to compare against.)

EDIT2: There’s another tutorial here about Ubuntu 11.04, Samba 3 and sharing with Windows.

Sometimes you want a raw log of whatever happens during a process, whether it’s coming in via STDERR or STDOUT (usually so you can see the warning or error in the context of the rest of the output). A little redirection and judicious use of “tee” will accomplish this. You may not know you wanted a log at all until you spent 5 hours running a process and an error occurred 20,000 lines back. Here’s a quick and easy way to log that output on the off-chance you’ll need it later!

Simple logging for bash (Linux or Cygwin) or a Windows command prompt (assumes you have “tee” installed somewhere in your path, e.g., via Cygwin). The “2>&1″ part redirects STDERR to STDOUT prior to piping STDOUT to “tee”:

MyUtil -arg1 A -arg2 B 2>&1 | tee mylogfile.log

Simple logging for csh/tcsh (Linux or Cygwin). The |& forces STDERR to be piped to “tee” alone with the usual STDOUT:

MyUtil -arg1 A -arg2 B      |& tee mylogfile.log

Note: non-interactive processes are far more amenable to this… if manual input is expected you probably won’t see the prompt (since tee won’t flush its buffer until EOF or a newline, at a minimum (it may buffer MANY lines before displaying them). If you know when it’ll occur you CAN enter the data at the appropriate time (or press return or what have you). This is a quick and dirty way to log something you’d rather not dig into to instrument.

Are YOU a Cygwin user who is not satisfied with GCC 3.4.4 (the latest one they offer)? Want to build a far more modern release of GCC for Cygwin? Read more on the Wikia – it works: I just spent a few hours (im)proving it!

(This also inspired my subsequent post about process logging… nothing makes good logging worthwhile like rebuilding a compiler!)

• Interesting software dev articles from EE Times:
  • “Top causes of nasty embedded software bugs”: Part 1 Part 2
  • “Bug-killing standards for firmware coding”: Part 1 Part 2

• 97 Things Every Programmer Should Know: An interesting little book that covers many aspects of the software development process (I have to add that you shouldn’t be afraid to break things unless that thing is the build. Don’t Break The Build!)

• Embedded C Coding Standard as envisioned by Netrino: A (quite expensive) book (the reviews I’ve read are a mixed bag as well). I’m not recommending it so much as pointing out that it exists and is well-regarded in some circles.

As an aside, while “don’t break the build” became my mantra early in my SCM roles, I had no idea how many variants there are on that theme on the web… everything from websites to t-shirts to videos. LOL!