Unlike a desktop app, where one could add a “safe mode” icon for such instances, there are few options in the mobile space. These include:

• Letting the user delete the application data manually via the Settings menu
• Automatically resetting to defaults after a crash
• Attempting to automatically work around the broken area
• Asking the user on restart if they want to reset to defaults
• Falling back to the last known-good configuration

It’s one thing to be a power user, but another to be forced to become one thus the first option isn’t really practical. The second option guarantees recovery but may degrade the user experience (especially if there are lots of settings involved)… it should be a last resort, and a user choice. The third option is nice, but can become quite complex: this works best if settings are checkpointed frequently, but such frequent saves to flash memory are not usually a great idea. In addition, the more complex the fail-safe, the more likely it may cause a triggering exception.

Asking the user first is an important part of any solution, but it requires architecting a fail-safe startup sequence (one that is preference invariant). One can then offer the choice of a “fresh start” or the option of using a backed-up configuration, ensuring that the user has at least a chance of restoring to a previously working state with their settings intact.

A useful pattern for a single-instance app would be the following, using a persistent IsRunning flag:

• App launches
• Load settings
• Check if the IsRunning flag is set: if it is, then the last run exited abnormally (none of the normal exit points were hit which would’ve cleared the flag):
• Clear the IsRunning flag
• Offer to restore from backup / defaults or attempt to continue as-is
• Initialize states and display initial UI presentation
• Save the current settings with a backup name
• Set the IsRunning flag
• Save the settings normally
• On normal app exit / backgrounding, clear the IsRunning flag and save settings normally

There are other things you can do, including instrumenting for crash data collection and presenting the user an option to forward that data (anonymized!) to you for analysis.

Software quality is as much about how well you avoid defects in the first place as it is about how gracefully you recover from them!^† Mistakes happen, especially in apps with complex GUIs. Errors should be as rare as possible, but when errors occur it’s the unrecoverable ones that lead to lost users. Solid error handling and recovery goes a long way towards satisfied users who stick around for the release that irons out the bug they encountered and worked around

^†This is a critical difference between hardware and software quality, especially for smaller shops. If you ship a physically defective device, the typical recourse is a return for repair or replacement – a costly proposition in terms of logistics as well as reworked hardware / waste. The quality patterns for software and hardware have commonalities but are generally quite different in practice: in general, hardware is by its very nature far more costly and less forgiving of quality failures. This is why popular hardware quality initiatives tend to translate poorly to the software space.

Consider apps that are meant to be full-screen, and may be deeply interactive (such as a game). Having the action bar onscreen is not tenable, so one hides it. Whereas before you could use the menu button in the gutter (where “home” and “back” reside and indeed where “menu” appears – only for legacy apps), now you must either code your own button or invent a unique (but intuitive!) gesture to accomplish the end result of opening a custom menu or showing the action bar. If that’s not UI fragmentation I don’t know what is!

Though I find the action bar awkwardly placed and non-intuitive for in-app interactions, I understand the merit of having a unified UI. What I find fault with is the shortsighted decision to effectively remove an important hard button from Android (does anyone miss the “search” button?)

Now we have only two guaranteed (not hideable) buttons left, for the purpose of app navigation: “back” and “home” – along with a third new button for fast task switching (which is a little redundant… long-press on the home button could easily have the same effect without adding buttons). In addition (and likely for the foreseeable future) we have a “menu” button that only appears for legacy apps (thus that space in the toolbar is already “reserved” and will remain so for quite some time).

It’s too bad… there’s a strong argument for always having a “menu” button handy in an always-visible place. Maybe it unhides your action bar, or maybe it opens a custom menu, but removing it doesn’t help matters: it simply encourages a new kind of fragmentation for apps that don’t need an action bar onscreen normally. Now, instead of having a consistent way to perform that “open menu” action everyone gets to do it their own way (or leave the action bar open full-time, which significantly detracts from some apps).

One of many things that I like about Android is that the basic controls (home, menu, back) are intuitive and reasonable, balancing function with minimalism. Contrast this with the one overloaded hard button in another popular OS: much like a one-button mouse, oversimplification leads to confusion, reduced functionality and a fragmented UI as everyone works around the limitation differently according to their app’s design.

The rationale for taking away the menu button – one of the three main “always-visible” navigation and control buttons for Android up to now – defies logic: it fails at reducing fragmentation and funnels developers (and thus users) into an awkward, one-size-fits-all paradigm that is the antithesis of Android. This loss of a “hard” menu button is an unwelcome reduction in basic functionality in an apparent nod to the oversimplification of other OSes. I don’t find fault with the existence of the action bar, but removing the dedicated menu button from the always-visible area of the screen was IMHO extremely shortsighted. Now, instead of having a convenient way to reveal a hidden action bar, one must figure out how to do this for each app that normally hides its action bar. All this for the sake of removing a button that appears anyway for legacy apps.

As part of this, I needed to rebuild the emulator. It was not as straightforward as I’d hoped so I decided to write up a little howto here in case others decide to try rebuilding it on Windows. It’s really not hard at all once you install a few prerequisites! [Note: You'll also find info here about building this via Linux.]

[Edit: As of this post the HEAD is 6e2f62...599648 for platform_external_qemu. While this post-r12 work WILL build in Windows with these procedures, I just found out that the older r12 code (tagged as android-sdk-tools_r12) will NOT build in Windows! You'd need to do r12-based builds in Linux with MinGW installed. It's interesting to see just how much was fixed since SDK r12... including, evidently, the very issue I discovered today! Hopefully SDK r13 gets released soon. Oh look, it's out! Interestingly, unlike r12 there is no corresponding r13 tag for this one... which is part of why I didn't notice it in the Git log (I saw the tools_r13 branch though there are lots of changes after that).]

Note: If you choose other install dirs, adjust the values below accordingly!

Install Python 2.6 or 2.7:

• I used ActivePython 2.7.2.5 x64 (x86 should be equally good)
• Download: http://www.activestate.com/activepython/downloads
• Default bin dir for Python = C:\Python27

Install MinGW (it includes msys now):

• Information: http://www.mingw.org/wiki/MinGW
• Download: http://sourceforge.net/projects/mingw/
• Default bin dir for MinGW = C:\MinGW\bin
• Default bin dir for msys = C:\MinGW\msys\1.0\bin

Install Git: I use msysgit because I also use (and happily recommend!) TortoiseGit 1.7.3.0:

• Information: http://code.google.com/p/msysgit/
• Download: http://code.google.com/p/msysgit/downloads/list
• Git-1.7.4-preview20110204.exe works, 1.7.6 may as well
• Default bin dir for Git = C:\Git\bin

Open a command prompt and ensure your path is set correctly (omit duplicates as necessary, but ensure such entries weren’t eclipsed by other things you’ve installed, like Cygwin):

set PATH=C:\MinGW\msys\1.0\bin;C:\MinGW\bin;C:\Python27;C:\Git\bin;%PATH%

Clone the repository! (Because of the kernel.org outage we’re using the github mirror):

mkdir H:\temp_repo
cd /D H:\temp_repo
git clone https://github.com/android/platform_external_qemu.git

Start bash in the folder with the emulator sources (from this point on commands run at the bash prompt):

cd /D H:\temp_repo\platform_external_qemu
bash

Setting PS1 makes the prompt more verbose than the default “bash-3.1$” prompt:

PS1='\[\e]0;\w\a\]\n\[\e[36m\]\u@\h \[\e[32m\]\w\[\e[0m\]\n\$ '

There’s a small problem if you want the version string in the emulator to be fully formed:

• In Makefile.common, ANDROID_BUILD_ID get set only if BUILD_STANDALONE_EMULATOR == “” – never if BUILD_STANDALONE_EMULATOR == ‘true’ (which is what we’re building here). Here’s a fix for that, at least for the 13.x line of the emulator (line numbers may vary!):
• Broken: line 66: ifeq ($(BUILD_STANDALONE_EMULATOR),)
• Patched: line 66: ifneq ($(BUILD_STANDALONE_EMULATOR),false)
• If in doubt, ignore this tweak! Understand your -version output will not be very informative. That can quickly get annoying if you are building multiple versions…

Set some variables and run the build:

# Customize this as desired
export ANDROID_SDK_TOOLS_REVISION=42
export ANDROID_BUILD_ID=custom

# Inserts your username and the timestamp in the build name
export MY_BUILD_TIMESTAMP=`date +%Y%m%d_%H%M%S`
export BUILD_ID=${USERNAME}
export BUILD_NUMBER=${MY_BUILD_TIMESTAMP}

# Executes the build
./android-rebuild.sh --static 2>&1 | \
   tee build_${USERNAME}-${MY_BUILD_TIMESTAMP}.log

Check the version string of the emulator you just built:

• Run this:
• objs/emulator.exe -version
• See output like this:
• Android emulator version 13.0 (build_id Marty-20110925_042430)

Using the newly-built emulator:

• The objects of interest in objs are:
• emulator.exe, emulator-arm.exe, emulator-x86.exe
• You may need to copy them to android-sdk-windows/tools folder to use them (back up the originals first before replacing them!)

Notes on building via Linux:

As mentioned above, I managed to get all this to work on Windows just fine… technically, no source edits are necessary at this point to make it work. However, when I switched to the android-sdk-tools_r12 tag it failed with:

cc1: error: unrecognized command line option “-mno-cygwin”

Turns out this is a known issue… and in fact it was fixed in a later commit to the emulator code base! (I saw it as I was browsing the overall delta between r12 and the latest.) But, even with that fixed I couldn’t get r12 to build within Windows, so I went to Linux. I was already configured to build CyanogenMod, so everything was in place except MinGW. I installed mingw32, made that one fix to android-rebuild.sh and ran:

./android-rebuild.sh –static –mingw

And it worked just fine!

(Note: the –mingw option is NOT used when building this within Windows… it’s a little confusing because of course we’re using MinGW in Windows, but that’s how it’s coded in their build system.)

This is all nice, but when you enable hybrid sleep you lose “hibernate” as a separate option. Perhaps this is to avoid confusion (remember: hybrid sleep performs hibernation as part of its process, it just doesn’t completely power down). However, there are times I’d rather have the system fully hibernate instead of just sleeping (such as if I’m going to be away a few days). Since touching the power button once it’s in full hybrid sleep often re-awakens my system (and I don’t prefer to power down directly from any sleep state), I wanted a one-click solution.

So, if you use hybrid sleep and want to explicitly hibernate, run this (even as a regular user) and your system should hibernate rather than just enter hybrid sleep:

%windir%\system32\rundll32.exe PowrProf.dll, SetSuspendState 0,1,0

This should also work if you DON’T use hybrid sleep and DO have the hibernation option… though if you have “hibernate” as an option you’re probably not looking for this sort of fix.

Disclaimers: If hibernation or hybrid sleep didn’t work before this won’t make hibernation work (e.g., if you have insufficient disk space or have explicitly disabled hibernation and hybrid sleep)… in that case this will likely send you into the typical sleep mode. You can usually tell the difference by noting the lengthy time spend writing data to the disk, followed by a clear power off. (Hybrid sleep does the same, except it goes into sleep mode – how your computer shows this will vary).

More info here.

2011-09-12 update: The version here is for Python 3.x as described below. Jens Diemer posted an updated version of this to GitHub with corrections. It’s mainly a backport of this script to Python 2.x along with some cosmetic changes (and yes, I do like delimiting functions and such with comment lines – it makes code blocks far easier to distinguish, even in indentation-intensive Python). He added a bugfix to handle SIGBREAK better, which I’ve pulled into the code below.

A while back, when I decided to learn Python, I looked into writing my own sort of ping utility with it. Then I learned just how ridiculously arduous it is to perform this seemingly-simple function (essentially, you have to be running as root in Unix (or Administrator in Windows) to accomplish this, and generally speaking running scripts like this as root is considered bad form). But it can be done…

This intrigued me as a pet project but I moved on to other things, eventually picking up Python along the way (I’ve focused on Python 3.x since the Unicode handling is FAR better than it is in 2.7, not to mention its other improvements).

Recently, I decided to revisit the “ping with Python” idea, and searched around to see if anyone else had tackled it. Sure enough, I found George Notaras’ site which has a very nice implementation in 2.x as well as a recent commenter who created a 3.x port. Only problem was, the HTML is mangled there so neither script worked. As I delved into it, I managed to get the scripts working, and then decided to go the extra mile and make some enhancements and improvements. Along the way I ended up getting the source for ping (via Cygwin) and building it from scratch as well (particularly to understand the checksum routine, and for other hints in general).

It was rather pleasing to not only get this working but to make it work far better in terms of functionality and robustness, all the while reinforcing and extending my understanding of Python.

Note: The original C source for “ping” can be found here (you’ll need to extract it from ping.shar). Or just search for ping.c… for example, I found an identical copy of ping.c here.

#!/usr/bin/env python
# -*- coding: utf-8 -*-

"""
    A pure python ping implementation using raw sockets.

    (This is Python 3 port of https://github.com/jedie/python-ping)

    Note that ICMP messages can only be sent from processes running as root
    (in Windows, you must run this script as 'Administrator').

    Derived from ping.c distributed in Linux's netkit. That code is
    copyright (c) 1989 by The Regents of the University of California.
    That code is in turn derived from code written by Mike Muuss of the
    US Army Ballistic Research Laboratory in December, 1983 and
    placed in the public domain. They have my thanks.

    Bugs are naturally mine. I'd be glad to hear about them. There are
    certainly word - size dependencies here.

    Copyright (c) Matthew Dixon Cowles, <http://www.visi.com/~mdc/>.
    Distributable under the terms of the GNU General Public License
    version 2. Provided with no warranties of any sort.

    Original Version from Matthew Dixon Cowles:
      -> ftp://ftp.visi.com/users/mdc/ping.py

    Rewrite by Jens Diemer:
      -> http://www.python-forum.de/post-69122.html#69122

    Rewrite by George Notaras:
      -> http://www.g-loaded.eu/2009/10/30/python-ping/

    Enhancements by Martin Falatic:
      -> http://www.falatic.com/index.php/39/pinging-with-python

    Revision history
    ~~~~~~~~~~~~~~~~

    October 12, 2011
    --------------
    Merged updates from the main project
      -> https://github.com/jedie/python-ping

    September 12, 2011
    --------------
    Bugfixes + cleanup by Jens Diemer
    Tested with Ubuntu + Windows 7

    September 6, 2011
    --------------
    Cleanup by Martin Falatic. Restored lost comments and docs. Improved
    functionality: constant time between pings, internal times consistently
    use milliseconds. Clarified annotations (e.g., in the checksum routine).
    Using unsigned data in IP & ICMP header pack/unpack unless otherwise
    necessary. Signal handling. Ping-style output formatting and stats.

    August 3, 2011
    --------------
    Ported to py3k by Zach Ware. Mostly done by 2to3; also minor changes to
    deal with bytes vs. string changes (no more ord() in checksum() because
    >source_string< is actually bytes, added .encode() to data in
    send_one_ping()).  That's about it.

    March 11, 2010
    --------------
    changes by Samuel Stauffer:
    - replaced time.clock with default_timer which is set to
      time.clock on windows and time.time on other systems.

    November 8, 2009
    ----------------
    Improved compatibility with GNU/Linux systems.

    Fixes by:
     * George Notaras -- http://www.g-loaded.eu
    Reported by:
     * Chris Hallman -- http://cdhallman.blogspot.com

    Changes in this release:
     - Re-use time.time() instead of time.clock(). The 2007 implementation
       worked only under Microsoft Windows. Failed on GNU/Linux.
       time.clock() behaves differently under the two OSes[1].

    [1] http://docs.python.org/library/time.html#time.clock

    May 30, 2007
    ------------
    little rewrite by Jens Diemer:
     -  change socket asterisk import to a normal import
     -  replace time.time() with time.clock()
     -  delete "return None" (or change to "return" only)
     -  in checksum() rename "str" to "source_string"

    December 4, 2000
    ----------------
    Changed the struct.pack() calls to pack the checksum and ID as
    unsigned. My thanks to Jerome Poincheval for the fix.

    November 22, 1997
    -----------------
    Initial hack. Doesn't do much, but rather than try to guess
    what features I (or others) will want in the future, I've only
    put in what I need now.

    December 16, 1997
    -----------------
    For some reason, the checksum bytes are in the wrong order when
    this is run under Solaris 2.X for SPARC but it works right under
    Linux x86. Since I don't know just what's wrong, I'll swap the
    bytes always and then do an htons().

    ===========================================================================
    IP header info from RFC791
      -> http://tools.ietf.org/html/rfc791)

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |Version|  IHL  |Type of Service|          Total Length         |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |         Identification        |Flags|      Fragment Offset    |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |  Time to Live |    Protocol   |         Header Checksum       |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                       Source Address                          |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                    Destination Address                        |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                    Options                    |    Padding    |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

    ===========================================================================
    ICMP Echo / Echo Reply Message header info from RFC792
      -> http://tools.ietf.org/html/rfc792

        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        |     Type      |     Code      |          Checksum             |
        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        |           Identifier          |        Sequence Number        |
        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        |     Data ...
        +-+-+-+-+-

    ===========================================================================
    ICMP parameter info:
      -> http://www.iana.org/assignments/icmp-parameters/icmp-parameters.xml

    ===========================================================================
    An example of ping's typical output:

    PING heise.de (193.99.144.80): 56 data bytes
    64 bytes from 193.99.144.80: icmp_seq=0 ttl=240 time=127 ms
    64 bytes from 193.99.144.80: icmp_seq=1 ttl=240 time=127 ms
    64 bytes from 193.99.144.80: icmp_seq=2 ttl=240 time=126 ms
    64 bytes from 193.99.144.80: icmp_seq=3 ttl=240 time=126 ms
    64 bytes from 193.99.144.80: icmp_seq=4 ttl=240 time=127 ms

    ----heise.de PING Statistics----
    5 packets transmitted, 5 packets received, 0.0% packet loss
    round-trip (ms)  min/avg/max/med = 126/127/127/127

    ===========================================================================
"""

#=============================================================================#
import os, sys, socket, struct, select, time, signal

if sys.platform == "win32":
    # On Windows, the best timer is time.clock()
    default_timer = time.clock
else:
    # On most other platforms the best timer is time.time()
    default_timer = time.time

#=============================================================================#
# ICMP parameters

ICMP_ECHOREPLY  =    0 # Echo reply (per RFC792)
ICMP_ECHO       =    8 # Echo request (per RFC792)
ICMP_MAX_RECV   = 2048 # Max size of incoming buffer

MAX_SLEEP = 1000

class MyStats:
    thisIP   = "0.0.0.0"
    pktsSent = 0
    pktsRcvd = 0
    minTime  = 999999999
    maxTime  = 0
    totTime  = 0
    fracLoss = 1.0

myStats = MyStats # Used globally

#=============================================================================#
def checksum(source_string):
    """
    A port of the functionality of in_cksum() from ping.c
    Ideally this would act on the string as a series of 16-bit ints (host
    packed), but this works.
    Network data is big-endian, hosts are typically little-endian
    """
    countTo = (int(len(source_string)/2))*2
    sum = 0
    count = 0

    # Handle bytes in pairs (decoding as short ints)
    loByte = 0
    hiByte = 0
    while count < countTo:
        if (sys.byteorder == "little"):
            loByte = source_string[count]
            hiByte = source_string[count + 1]
        else:
            loByte = source_string[count + 1]
            hiByte = source_string[count]
        sum = sum + (hiByte * 256 + loByte)
        count += 2

    # Handle last byte if applicable (odd-number of bytes)
    # Endianness should be irrelevant in this case
    if countTo < len(source_string): # Check for odd length
        loByte = source_string[len(source_string)-1]
        sum += loByte

    sum &= 0xffffffff # Truncate sum to 32 bits (a variance from ping.c, which
                      # uses signed ints, but overflow is unlikely in ping)

    sum = (sum >> 16) + (sum & 0xffff)    # Add high 16 bits to low 16 bits
    sum += (sum >> 16)                    # Add carry from above (if any)
    answer = ~sum & 0xffff              # Invert and truncate to 16 bits
    answer = socket.htons(answer)

    return answer

#=============================================================================#
def do_one(destIP, timeout, mySeqNumber, numDataBytes):
    """
    Returns either the delay (in ms) or None on timeout.
    """
    global myStats

    delay = None

    try: # One could use UDP here, but it's obscure
        mySocket = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.getprotobyname("icmp"))
    except socket.error as e:
        print("failed. (socket error: '%s')" % e.args[1])
        raise # raise the original error

    my_ID = os.getpid() & 0xFFFF

    sentTime = send_one_ping(mySocket, destIP, my_ID, mySeqNumber, numDataBytes)
    if sentTime == None:
        mySocket.close()
        return delay

    myStats.pktsSent += 1;

    recvTime, dataSize, iphSrcIP, icmpSeqNumber, iphTTL = receive_one_ping(mySocket, my_ID, timeout)

    mySocket.close()

    if recvTime:
        delay = (recvTime-sentTime)*1000
        print("%d bytes from %s: icmp_seq=%d ttl=%d time=%d ms" % (
            dataSize, socket.inet_ntoa(struct.pack("!I", iphSrcIP)), icmpSeqNumber, iphTTL, delay)
        )
        myStats.pktsRcvd += 1;
        myStats.totTime += delay
        if myStats.minTime > delay:
            myStats.minTime = delay
        if myStats.maxTime < delay:
            myStats.maxTime = delay
    else:
        delay = None
        print("Request timed out.")

    return delay

#=============================================================================#
def send_one_ping(mySocket, destIP, myID, mySeqNumber, numDataBytes):
    """
    Send one ping to the given >destIP<.
    """
    destIP  =  socket.gethostbyname(destIP)

    # Header is type (8), code (8), checksum (16), id (16), sequence (16)
    myChecksum = 0

    # Make a dummy heder with a 0 checksum.
    header = struct.pack(
        "!BBHHH", ICMP_ECHO, 0, myChecksum, myID, mySeqNumber
    )

    padBytes = []
    startVal = 0x42
    for i in range(startVal, startVal + (numDataBytes)):
        padBytes += [(i & 0xff)]  # Keep chars in the 0-255 range
    data = bytes(padBytes)

    # Calculate the checksum on the data and the dummy header.
    myChecksum = checksum(header + data) # Checksum is in network order

    # Now that we have the right checksum, we put that in. It's just easier
    # to make up a new header than to stuff it into the dummy.
    header = struct.pack(
        "!BBHHH", ICMP_ECHO, 0, myChecksum, myID, mySeqNumber
    )

    packet = header + data

    sendTime = time.time()

    try:
        mySocket.sendto(packet, (destIP, 1)) # Port number is irrelevant for ICMP
    except socket.error as e:
        print("General failure (%s)" % (e.args[1]))
        return

    return sendTime

#=============================================================================#
def receive_one_ping(mySocket, myID, timeout):
    """
    Receive the ping from the socket. Timeout = in ms
    """
    timeLeft = timeout/1000

    while True: # Loop while waiting for packet or timeout
        startedSelect = time.time()
        whatReady = select.select([mySocket], [], [], timeLeft)
        howLongInSelect = (time.time() - startedSelect)
        if whatReady[0] == []: # Timeout
            return None, 0, 0, 0, 0

        timeReceived = time.time()

        recPacket, addr = mySocket.recvfrom(ICMP_MAX_RECV)

        ipHeader = recPacket[:20]
        iphVersion, iphTypeOfSvc, iphLength, \
        iphID, iphFlags, iphTTL, iphProtocol, \
        iphChecksum, iphSrcIP, iphDestIP = struct.unpack(
            "!BBHHHBBHII", ipHeader
        )

        icmpHeader = recPacket[20:28]
        icmpType, icmpCode, icmpChecksum, \
        icmpPacketID, icmpSeqNumber = struct.unpack(
            "!BBHHH", icmpHeader
        )

        if icmpPacketID == myID: # Our packet
            dataSize = len(recPacket) - 28
            return timeReceived, dataSize, iphSrcIP, icmpSeqNumber, iphTTL

        timeLeft = timeLeft - howLongInSelect
        if timeLeft <= 0:
            return None, 0, 0, 0, 0

#=============================================================================#
def dump_stats():
    """
    Show stats when pings are done
    """
    global myStats

    print("\n----%s PYTHON PING Statistics----" % (myStats.thisIP))

    if myStats.pktsSent > 0:
        myStats.fracLoss = (myStats.pktsSent - myStats.pktsRcvd)/myStats.pktsSent

    print("%d packets transmitted, %d packets received, %0.1f%% packet loss" % (
        myStats.pktsSent, myStats.pktsRcvd, 100.0 * myStats.fracLoss
    ))

    if myStats.pktsRcvd > 0:
        print("round-trip (ms)  min/avg/max = %d/%0.1f/%d" % (
            myStats.minTime, myStats.totTime/myStats.pktsRcvd, myStats.maxTime
        ))

    print()
    return

#=============================================================================#
def signal_handler(signum, frame):
    """
    Handle exit via signals
    """
    dump_stats()
    print("\n(Terminated with signal %d)\n" % (signum))
    sys.exit(0)

#=============================================================================#
def verbose_ping(hostname, timeout = 1000, count = 3, numDataBytes = 55):
    """
    Send >count< ping to >destIP< with the given >timeout< and display
    the result.
    """
    global myStats

    signal.signal(signal.SIGINT, signal_handler)   # Handle Ctrl-C
    if hasattr(signal, "SIGBREAK"):
        # Handle Ctrl-Break e.g. under Windows
        signal.signal(signal.SIGBREAK, signal_handler)

    myStats = MyStats() # Reset the stats

    mySeqNumber = 0 # Starting value

    try:
        destIP = socket.gethostbyname(hostname)
        print("\nPYTHON PING %s (%s): %d data bytes" % (hostname, destIP, numDataBytes))
    except socket.gaierror as e:
        print("\nPYTHON PING: Unknown host: %s (%s)" % (hostname, e.args[1]))
        print()
        return

    myStats.thisIP = destIP

    for i in range(count):
        delay = do_one(destIP, timeout, mySeqNumber, numDataBytes)

        if delay == None:
            delay = 0

        mySeqNumber += 1

        # Pause for the remainder of the MAX_SLEEP period (if applicable)
        if (MAX_SLEEP > delay):
            time.sleep((MAX_SLEEP - delay)/1000)

    dump_stats()

#=============================================================================#
if __name__ == '__main__':

    # These should work:
    verbose_ping("heise.de")
    verbose_ping("google.com")

    # Inconsistent on Windows w/ ActivePython (Python 3.2 resolves correctly
    # to the local host, but 2.7 tries to resolve to the local *gateway*)
    verbose_ping("localhost")

    # Should fail with 'getaddrinfo failed':
    verbose_ping("foobar_url.foobar")

    # Should fail (timeout), but it depends on the local network:
    verbose_ping("192.168.255.254")

    # Should fails with 'The requested address is not valid in its context':
    verbose_ping("0.0.0.0")

#=============================================================================#

Recently a friend on Twitter sent a link as part of a message:

“Look! it’s you in this picture.. [along with a tinyurl link]”

Other variants of this exist, such as “you’ll laugh when you see this pic of you… [along with a tinyurl link]“.

Turns out, my friend’s account had been hijacked. (Yes, it really was a friend… I’ve seen this before with Twitter and other sites but it looks identical to an attack I saw about a month ago).

In this case, the bogus link went to a subpage on itwittiler.com (IP address 220.164.140.252), a domain registered in China - earlier today! The page looks like Twitter’s login page but it’s not. In fact, I sandboxed it and entered a bogus username/password to see if it’d redirect to Twitter no matter what was entered. No… it redirected me to a “StalkTrak” page, clearly malformed and totally bogus. Read this Sophos security article for more info.

Another subpage (found via some searching online) does redirect to Twitter, but to a user who doesn’t exist. In all cases, the first page you go to looks just like the Twitter login page (there are some subtle yet sloppy differences, but next time there might not be.)

It’s quite likely the page is harvesting credentials along the way. If you get stung, go to the real Twitter homepage and reset your password pronto (after closing your browser windows).

More info on itwittiler.com, via whois:

Domain Name: ITWITTILER.COM
Registrar: JIANGSU BANGNING SCIENCE & TECHNOLOGY CO. LTD
Whois Server: whois.55hl.com
Referral URL: http://www.55hl.com
Name Server: DNS5.4CUN.COM
Name Server: DNS6.4CUN.COM
Status: ok
Updated Date: 13-aug-2011
Creation Date: 13-aug-2011
Expiration Date: 13-aug-2012

Update: Another StalkTrak spam domain made the rounds last month: itiwitter.com. Surprise! It’s registered to the very same registrar! Judging by the same subpage structure I strongly suspect it’s the very same scammer using the very same site (or site structure)… while the IP address won’t resolve, it turns out this site’s for sale! And the IP address when last they checked? 220.164.140.252. Isn’t that special?

Domain Name: ITIWITTER.COM
Registrar: JIANGSU BANGNING SCIENCE & TECHNOLOGY CO. LTD
Whois Server: whois.55hl.com
Referral URL: http://www.55hl.com
Name Server: DNS5.4CUN.COM
Name Server: DNS6.4CUN.COM
Status: clientHold
Status: clientTransferProhibited
Updated Date: 20-jul-2011
Creation Date: 15-jul-2011
Expiration Date: 15-jul-2012

Still more information can be found on this site, which talks about the same scam (back before the scammers changed domain names and came back from the “dead”).

For web/email hosting, I was with Concentric way back when (later bought by XO.com). I decided to leave XO.com when it was clearly unresponsive to the fact that the shared SMTP address was starting to routinely get blacklisted because spammers were hosted on the same machine my sites were unfortunately on. Instead of dealing with the spammers the situation was left as-is and was untenable.

I looked around and found Hosting Matters was recommended by people at the time. You can read about them elsewhere, for better or worse. They tended to be pretty responsive but I got tired of the service interruptions on the hosts I was on. Perhaps I was just unlucky that every server I was on ended up overloaded or DDoSed, but after moving servers two or three times it got old.

I also learned much later that Hosting Matters is also well-known for hosting right-wing / highly conservative websites (apparently this is why they were getting attacked frequently and why they were pretty good at fending off such attacks). This wasn’t a good fit for me either – I prefer my host to be if anything centrist/neutral, not a lightning rod.

After the latest little outage I decided to bite the bullet and migrate to HostGator. It’s also CPanel-based and has most of the same features I was used to (I’m happy to be able to custom configure SpamAssassin again, and the resource-hogging mailscanner is absent which may also be good). Otherwise the features are almost all identical or slightly better (apparently Hosting Matters had automatic backups I was unaware of, which I’m not seeing on HostGator, but that also might be why my Hosting Matters server kept bogging down late at night).

I’m still getting used to it but already there’s at least two clear differences: the server seems remarkably faster (both in general bandwidth as well as in low load) and I was able to update WordPress and its plugins directly from the WordPress admin panels (something I was never able to do on the previous host). Considering I simply exported and imported the databases and then created/extracted the tarball of the full sites (including the WordPress installations) I was totally shocked that it just worked (I gave it a shot thinking it’d fail and I’d investigate what might be wrong…) The relevant permissions and groups seem the same though.

Note: I didn’t use HostGator’s migration service… instead, I configured and migrated it all by hand (for the experience).

The HostGator bottom line: pleasantly surprised and satisfied so far!

For VPS, I was with Slicehost for over two years with a standard single slice. It was interesting and I liked the interface, but their recent acquisition by Rackspace was followed by an announcement that Rackspace will discontinue the Slicehost VPS offering in favor of their Cloud offering. I may go back to them later just to try out the whole cloud computing paradigm, but that isn’t really suitable for my VPS needs.

Seeing a lot of people migrating to Linode, I looked into it. For the same money I get more space and memory as well as the ability to use StackScripts, which should make deploying or rebuilding a server as easy as clicking a button.

I used my tried-and-true cheat sheet (which I previously used with my Slicehost slice) to create a new Ubuntu 11.04 x64 linode. I had to make a couple of minor changes (I was previously on Ubuntu 10.10) but otherwise my new Linode worked flawlessly!

The Linode bottom line: satisfied with the better value and knowing that it’s not disappearing anytime soon.

** I’ve pondered moving all my hosting to that VPS, but it doesn’t make sense, particularly for the email part. (I’m not thrilled with the idea of outsourcing my email to a third party either.) It’s a ton of maintenance and upkeep which shared hosting avoids. Tradeoff is potential partial or full outages, but hopefully those stay rare.

First, the tail of the installation output for the samba4 package:

…
module samba_dsdb initialization failed : No such object
Unable to load modules for /var/lib/samba/private/sam.ldb: dsdb_module_search_dn: did not find base dn @ROOTDSE (0 results)
Traceback (most recent call last):
File “/usr/share/samba/setup/upgradeprovision”, line 1597, in <module>
ldbs = get_ldbs(paths, creds, session, lp)
File “/usr/lib/python2.7/dist-packages/samba/upgradehelpers.py”, line 159, in get_ldbs
ldbs.sam = SamDB(paths.samdb, session_info=session, credentials=creds, lp=lp, options=["modules:samba_dsdb"])
File “/usr/lib/python2.7/dist-packages/samba/samdb.py”, line 53, in __init__
options=options)
File “/usr/lib/python2.7/dist-packages/samba/__init__.py”, line 110, in __init__
self.connect(url, flags, options)
File “/usr/lib/python2.7/dist-packages/samba/samdb.py”, line 66, in connect
options=options)
_ldb.LdbError: (80, ‘dsdb_module_search_dn: did not find base dn @ROOTDSE (0 results)’)
dpkg: error processing samba4 (–configure):
subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
samba4
E: Sub-process /usr/bin/dpkg returned an error code (1)

I first looked around… a few people had reported similar issues but the answers weren’t useful. The most common one in several places was, “Why are you using samba 4? It’s experimental!” I’ve heard that only samba4 works (well | at all?) with Windows 7 protocols. Not sure about that but it was clear that downgrading to regular samba (v3) wasn’t an ideal solution. (Note: it’s been a while since I dealt with interop issues on samba… I apparently originally installed samba4 in 10.04 or 10.10 as part of trying to talk with Windows 7.)

Finally, having done due diligence, I posted my problem as a bug report. After a few iterations it was mentioned that I should “purge” the samba database. Hmm. I didn’t see that mentioned anywhere else when I searched before (on my particular install error). So I tried it and lo and behold! The purge worked!

So, to install/upgrade samba4 on Ubuntu 11.04, you can try the following (Disclaimer: always back up first!):

# Attempt to install samba4:

sudo apt-get install samba4

# Assuming it fails to install correctly
#("_ldb.LdbError: (80, 'dsdb_module_search_dn: did not find
# base dn @ROOTDSE (0 results)')" is the error I was dealing
# with), purge samba4:

sudo apt-get purge samba4

# Finally, attempt to install samba4 again:

sudo apt-get install samba4

That should do it. If it doesn’t do it, maybe you are having a different issue.

EDIT: There’s some useful info on this page about configuring Ubuntu 11.04 Samba so as to play nice with Windows. Also, your smb.conf is probably in /etc/samba… installing samba doesn’t install a default config file even if this one is absent. I’m not sure why yet but I’d like to know since my existing one is throwing warnings when you run “samba” as a command (several “unknown parameters” are getting “ignored”… probably fine but I wish I had a clean smb.conf for this release to compare against.)

EDIT2: There’s another tutorial here about Ubuntu 11.04, Samba 3 and sharing with Windows.

Sometimes you want a raw log of whatever happens during a process, whether it’s coming in via STDERR or STDOUT (usually so you can see the warning or error in the context of the rest of the output). A little redirection and judicious use of “tee” will accomplish this. You may not know you wanted a log at all until you spent 5 hours running a process and an error occurred 20,000 lines back. Here’s a quick and easy way to log that output on the off-chance you’ll need it later!

Simple logging for bash (Linux or Cygwin) or a Windows command prompt (assumes you have “tee” installed somewhere in your path, e.g., via Cygwin). The “2>&1″ part redirects STDERR to STDOUT prior to piping STDOUT to “tee”:

MyUtil -arg1 A -arg2 B 2>&1 | tee mylogfile.log

Simple logging for csh/tcsh (Linux or Cygwin). The |& forces STDERR to be piped to “tee” alone with the usual STDOUT:

MyUtil -arg1 A -arg2 B      |& tee mylogfile.log

Note: non-interactive processes are far more amenable to this… if manual input is expected you probably won’t see the prompt (since tee won’t flush its buffer until EOF or a newline, at a minimum (it may buffer MANY lines before displaying them). If you know when it’ll occur you CAN enter the data at the appropriate time (or press return or what have you). This is a quick and dirty way to log something you’d rather not dig into to instrument.

Are YOU a Cygwin user who is not satisfied with GCC 3.4.4 (the latest one they offer)? Want to build a far more modern release of GCC for Cygwin? Read more on the Wikia – it works: I just spent a few hours (im)proving it!

(This also inspired my subsequent post about process logging… nothing makes good logging worthwhile like rebuilding a compiler!)

• Interesting software dev articles from EE Times:
  • “Top causes of nasty embedded software bugs”: Part 1 Part 2
  • “Bug-killing standards for firmware coding”: Part 1 Part 2

• 97 Things Every Programmer Should Know: An interesting little book that covers many aspects of the software development process (I have to add that you shouldn’t be afraid to break things unless that thing is the build. Don’t Break The Build!)

• Embedded C Coding Standard as envisioned by Netrino: A (quite expensive) book (the reviews I’ve read are a mixed bag as well). I’m not recommending it so much as pointing out that it exists and is well-regarded in some circles.

As an aside, while “don’t break the build” became my mantra early in my SCM roles, I had no idea how many variants there are on that theme on the web… everything from websites to t-shirts to videos. LOL!

What’s also very interesting is the aftermarket trade in all this.  As others point out, articles like this are not how criminals learn of such things – I’m sure there’s outfits out there buying up old printers, copiers or just the raw drives themselves, selectively, simply to vacuum useful info from them.

Why encryption and wipe-on-completion isn’t mandated is a mystery to me.  An exec suggested it’s a $500 add-on for one company’s products, but to me it should be a fundamental part of the product’s design!  Is it so hard to at least write a routine that overwrites a used/deleted file?  Not at all!  A big, fat “factory wipe” option loudly proclaimed in the manual?  A fraction of a cent’s worth of ink.

Note: encryption would be a good thing, but doing it right would add to hardware and design costs – otherwise the keys (being locally stored) are not much more secure than the data.  It’s far simpler to implement wipe-after-use and wipe-nightly.

See also the CBS video of this:

Update 1: This procedure works on the zxing-1.4.zip source base.  I’m trying to figure out why it doesn’t want to work easily with the most current SVN code (e.g., rev 1217 or 1220).  I thought it was just some preloading weirdness that caused the Java Stack Overflow errors, then I thought it was Subversion related, but now… now I’m not sure what’s going on but I’ll let you know if I figure it out.

Update 2: Eclipse does NOT like the new version of /android/default.properties at all.  After you create the project folders but before you try creating the project in Eclipse, edit /android/default.properties and change

target=android-4

to

target=Google Inc.:Google APIs:4

That seems to prevent the whole Stack Overflow problem.  I’m not sure why this changed or what the more correct thing to do it but there’s some apparently relevant discussion here. Curiously, despite my reversion of this change I do not see any problem in my test builds here, whether in Eclipse or when using the command-line build as outlined previously.

Update 3: If you comment out the target= line in that file entirely (using “#”) the problem also goes away – be sure to select an SDK level >= 4 though.  Eclipse still forces you to select a Google APIs target, but if you double-click one of the “un-selectable” Android targets (click the text NOT the checkbox!) just right it’ll apparently take the setting and work OK.  Who knows if that works in general – probably not. With an Android target launching the AVD also goes much more smoothly!

Question is, what is it that makes Eclipse insist this is an APIs project when default.properties is not giving it any hints? Or is it a quirk of the SDK? What triggers this, and is it more readily resolved than by doing this?

Update 4: Last update for tonight. Once a project is in Eclipse you can go to Properties -> Android for that project and select a suitable Android target.  As long as you don’t have dependencies on Google APIs it should be fine.  If you /android/AndroidManifest.xml and remove the android:minSdkVersion setting that problem goes away as well (again, select Android 1.6 SDK level 4 to ensure proper operation).

Either it’s Eclipse 3.5.1 or the ADT plugin that’s messing up here.  Regardless, it doesn’t seem to be any flaw in ZXing.  It’s rather annoying to see Eclipse acting like this… are other people seeing this sort of bug?

To summarize: to avoid all these painful problems just plain remove default.properties and edit AndroidManifest.xml before you create a new Eclipse project based on the latest source.  Ensure you select Android 1.6 SDK level 4 for best results.

Also, when you run the app it’s best to just start the AVD first, wiping user data, (or at least uninstalling the old version of your app) and being at the home screen before you “Run As”.  Otherwise, the app’s display ends up rather funky.

Finally, “Build Automatically” may be more trouble than it’s worth, at least for a larger project such as this.  Consider this if your head starts making dents in the wall as you struggle to make sense of Eclipse’s random “out of memory” errors.

In my previous post I referred to a Knol that tells you how to build ZXing for Android (the Barcode Scanner) using the Eclipse IDE.  There are pros and cons to doing this, given that ZXing is normally built from the command line using Ant (and during that build it is optimized using Proguard).

However, my goal is to learn more about Android programming in general and to make it even more interesting, I’m doing it from Windows.  Having the project working in an IDE is preferable for me when it comes to unraveling functional connections and learning how a (quite useful!) Android app is put together.

First of all, that Knol isn’t all that bad.  It helps if you read it carefully (I kept screwing up a particular step), however some of the steps are unclear apparently because the SDK has changed rather substantially since it was first written.  I’ve tried to be ridiculously detailed here so as to eliminate any possible confusion for the novice.

Second, it’s not clear that you need to install anything other than the Eclipse Classic 3.5.1 IDE and the Android SDK to perform this experiment.  I have the JDK, Ant and Proguard as well but they aren’t in my system paths nor is JAVA_HOME set system-wide yet.  Therefore I suspect none of that other stuff is getting referenced in this experiment.  You should have the IDE installed, the Android SDK installed and up-to-date, and the ADT Plugin for Eclipse (found on the SDK page).

Initialize the project in Eclipse

• Obtain the ZXing source tree via zip or SVN as described in the previous posts.  We are only interested in the /android and the /core/src trees.  The rest of the paths and components may be ignored for the purpose of building the Barcode Scanner app for Android.
• Place these in a working area.  For this example I used D:\workspace\zxing
• An important note: the Android SDK Content loader doesn’t immediately start in Eclipse until you attempt to use or create an Adroid project, and it takes a while to load.   Therefore quickly creating a new Android project based on another Android project while the SDK is loading can have unwanted consequences – particularly, Stack Overflows – which is apparently due to a race condition (if the Android SDK Content Loader is still processing when you pick the “based on” tree, bad, bad things happen).  It was a very distracting problem until I realized what was going on.
• Start the Eclipse IDE
  • Close any open projects!
  • Note that Project -> Build Automatically is selected
• From the menu select File -> New -> Project…
• In the new window select Android -> Android Project
• If Build targets are NOT populated at all and there’s activity in the progress tab:
  • Close the New Project window.
  • Wait for the Android SDK Content Loader activity to complete (watch the progress tab!)
  • From the menu select File -> New -> Project… again
  • In the new window select Android -> Android Project again
  • At this point the Build Targets should be properly populated and the progress tab should be idle/empty.
• Give the project a name.  For this example I used “BarExample” and will refer to that going forward.
• Select Create project from existing source
• Browse to where you put the zxing source tree (this will become your workspace!)
• Select the /android subfolder and click OK
• The Build Target will automatically be selected as Google APIs 1.6
  • I’m not entirely sure why this happens – you cannot select Android alone either.  I think this is why you are later warned when trying to launch it that there are no compatible devices.
• The Application name will be auto-filled as CaptureActivity
• Click Finish

Allow it to prebuild.  At this point you’ll have some 250+ errors but the next step should resolve them!

Add the /core/src code to your project:

Hard way #1:

Hard way #2:

A somewhat easier way to do this (avoiding the extra copy) is to put /core/src under /android (thus creating /android/core/src) before you initialize the project!  Then, after you initialize the project you can pick the src folder under the core entry in the project items and select Build Path -> Use as a Source Folder to have the desired effect (it’ll crete an entry called core/src).  I would expect a symlink, correctly applied, would also have the desired effect.  As you can see, the android folder itself is really superfluous here.  [I still think there's got to be an even easier way to gather this all together, without moving ANYTHING around.]

The best way to add /core/src to your project! No symlinks required (and being Windows it’s possible but not easy and besides this is even more minimalist) and it should work seamlessly even if you’re doing thing with Subversion-controlled code.

You’ve already created a project area called zxing (or whatever) and in it are the /android tree and the /core/src tree. Rather than manually or automatically move ANY code, simply do this:

• In the Package Exp tab, right click on the project name (“BarExample”) and select Build Path -> Link Source…
• Browse to and select the src folder in core.
• Change Folder name to srccore (or any other not-in-use folder name you prefer).
• Select Finish
• Let it rebuild automagically

If you were successful, you should see a whole mess of warnings (I see over 170) – but no errors!

To install and test this on the Android Virtual Device:

• Go to the Android SDK and AVD Manager and start your test device
  • I previously created a Droid-compatible device configured as Platform 2.0.1 API Level 6, with a WVGA854 display and a 32M SD card.
  • Select the device and click Start…
  • To avoid weirdness I elected to run with a clean slate and so I selected the “Wipe user data” option
  • The virtual device should start up.
  • You can (and probably should) close the AVD Manager at this point. Leaving the manager on led to an odd “GC overhead limit exceeded” error.
  • Wait patiently for the main screen to appear (it seems to take quite a while and a lot of CPU – is it just me?)
  • You shouldn’t have any other versions of this Barcode app installed at this point.  If you DO you should probably uninstall them first!
• Now, back to Eclipse!
  • In your project, select Run As -> Android Application
  • At this point I get an AVD error telling me “No compatible targets were found”.
  • I selected “No” (I don’t want to create a new target – I already have one!)
  • In the Device Chooser that follows I selected my running 2.0.1 emulator and clicked OK.
  • (I don’t know why this happens – it’s probably the mismatch between my 2.0.1. AVD and the original APIs 1.6 target.)
• The app should upload, install and execute on the virtual device.
• Be sure to rotate your local gravitational reference clockwise by 90° for the optimum viewing experience.  See the January, 2153 quantum multicast of Make: The Virtual Magazine for info on how to construct a portable graviton generator for this purpose.
• It’s not terribly interesting at this point but should work and look normal (not wiping user data and/or letting it launch the app before the screen is “unlocked” had something to do with the display looking rather weird – I’m not sure what was going wrong).
• (Really, why doesn’t the AVD have a rotate function?  It seems pretty fundamental.  Likewise some way to fake camera input – do tell if I’m missing a simple way to do either!)

Like I’ve mentioned, I got various odd little IDE errors as I made my way through all this… perhaps I’m doing something that’s not entirely right or the Eclipse IDE / Android SDK combination isn’t very stable.

But will it blend?  That is, will it work on a physical device?  Heck yeah!

To install and test this on an Android physical device:

• First, uninstall the current Barcode Scanner from your phone (you shouldn’t need to do it each time you cycle through your private test builds, but you DEFINITELY need to if you are switching between the officially signed to the unsigned or debug signed versions!  In short: if you get an installation error uninstall the app and try intalling it again!
• Note: for some bizarre reason after installation the Barcode app may be out of order alphabetically in the Manage applications dialog!  Closing the dialog and going back to it via Settings rectifies this.
• Copy the application’s APK from /android/bin to your phone
• Install the application
• Run the Barcode Scanner!

That’s about it!  I’d expect this to work just as well from a Linux host but I haven’t tested it.

Caveats:

• Proguard isn’t being used in this configuration AFAIK.  Therefore, things aren’t optimized (I’m not yet clear on why that’s necessary – presumably for performance reasons though it seemed to work OK when I tried it).
• AFAIK this is NOT how the ZXing project is normally developed, so if you plan to submit changes to it or otherwise assist with the project you should defer to the project’s standards and procedures.
• This procedure worked fine for me but it may or may not work as well for you, and there may be unexpected side-effects (I haven’t seen any yet but you never know).

It went very smoothly! Here’s the details:

• Install the TortoiseSVN client (then reboot)

• Create a folder to work in (e.g., “D:\zxing-test”)

• Right-click in/on the folder and select “SVN Checkout…”
  - Edit the “Checkout Directory” accordingly
  - I used the info at the ZXing SVN page to get the required URL (http://zxing.googlecode.com/svn/trunk/).  Note that the “zxing-read-only” part caused problems here (it’s not valid syntax for this client). You cannot commit changes without being a member of the group so leave it off when using TortoiseSVN.
  - I did a fully recursive code pull the first time.
  - I selected the HEAD revision of the version tree.

• Time passes as you download quite a helluva lot of code!  Broadband helps…

• For this experiment the last commit version turned out to be revision 1217.  I’m not sure how they manage their changes but it’s always possible you could be unlucky and get a revision level that’ll lead to a broken build.  (Fortunately, tonight’s my lucky night!)  You can specify this when you start the SVN interaction above, or you can “revert” to this version (or any other) later if you prefer.

• Prep and build it!
  - Edit build.properties as I described in my last post.
  - Build the source tree (be sure to set MY_PROJ_ROOT correctly to point to the SVN-derived tree!)  Use the debug target as before (I’ve yet to try the release one though I suspect the lack of signing will cause problems).

And again, voilà!  You’ve got a new build! (I notice the first “clean” build step no longer throws a small error as it did before… it didn’t affect things but it was nice to see that it got fixed).

I loaded the build onto my Droid and installed it (there was no need to uninstall the older version 3.0 I was testing with earlier – I don’t bother with that unless it complains or otherwise fails to install).  On execution the about box shows this is version 3.2 beta 1 (versus the version 3.0 I built earlier and version 3.1 which was the most current in the Android App Market)!  If I’m going to fiddle around, it’s nice to know I’m fiddling with the latest and greatest!

That concludes my coding evening.  Thanks for visiting!

Note: Other projects use Git… I installed TortoiseGit as well but haven’t had a chance to play with it yet.

Tonight I decided to try my hand again at building an Android app – this time, one that should in theory be easy and yet feature-rich enough to learn from.  It was far from the simple experiment it ought to have been, so I’m sharing my experience to hopefully save you the reader some time if you try this yourself!

ZXing is the heart of the Barcode Scanner app for Android.  It’s nifty though it doesn’t take full advantage of my Droid’s camera resolution and there are some other tricks I’d like to see it do (such as activate the flash for improved lighting, replacing the constant autofocus with a forced focus, and perhaps adding some decoding protocols that aren’t supported in the Droid build of this, at least as of Barcode Scanner v3.1).  My main goal was to simply rebuild the app for the Android target via a Windows host, then play around with the code to learn more about Android coding.

I’d recently moved some install folders around so I decided to install the necessary parts from scratch.  Note that I first spent an hour or two trying to use Eclipse 3.5.1 to build this but haven’t gotten it to work (yet).  When I decided to try doing this via “basic principles” it quickly became apparent that this would take a while.

I ran into several issues I won’t bore you with here.  Suffice to say the answers are scattered around Google, either in the ZXing comments, discussions, or its bugs area, and sometimes completely elsewhere.  It became clear that while Windows is supported as a host it’s probably not the primary host they test builds from, so their instructions aren’t very current or coherent for it needed some tweaks, which they’ve made.  (Windows isn’t the most common way to build for Android but it’s quite do-able just the same.)

Here, then, is the distillation of what I learned in order to successfully build the Barcode Scanner app for Android via Windows. This should apply to other Windows versions as well, at least back to XP.  If you are using an x86-based Windows then use x86 installers instead of x64 installers (where there’s a difference).

# This is the setup for a Windows 7 host (64-bit on an Intel CPU)

# Install Sun JDK (jdk-6u18-windows-x##):
 - Source: http://java.sun.com/javase/downloads/index.jsp
    - Select the JDK appropriate to your Windows host (x64 or x86)
 - My root folder: "C:\Program Files\Java\jdk1.6.0_18"

# Install Sun WTK 2.5.2 (sun_java_wireless_toolkit-2.5.2_01-win.exe):
 - Yes, you need this too!
 - Source: http://java.sun.com/products/sjwtoolkit/
    - Select Sun Java Wireless Toolkit 2.5.2_01 for CLDC
 - My root folder: "C:\WTK2.5.2_01"

# WTK is NOT required for the Android target, but part of its folder structure IS, as follows.

# Install ProGuard Java Optimizer (proguard4.4.zip):
 - Source: http://sourceforge.net/projects/proguard/
 - My root folder: "D:\Programming\proguard4.4"

# Copy the proguard.jar file to the expected location in WTK:
 copy D:\Programming\proguard4.4\lib\proguard.jar C:\WTK2.5.2_01\bin
 - If the "C:\WTK2.5.2_01\bin" tree doesn't exist, simply create it and populate it as above
    (again, WTK itself is not required for the Android target)

# Install Apache Ant (apache-ant-1.8.0-bin):
 - Source: http://ant.apache.org/bindownload.cgi
 - My root folder: "D:\Programming\apache-ant-1.8.0"

# Install Google Android SDK (android-sdk_r04-windows):
 - Source: http://developer.android.com/sdk/index.html
 - My root folder: "D:\Programming\android-sdk-windows"

# Run "android-sdk-windows/SDK Setup":
 - Update all packages!

# Fix "dx.bat" for Android SDK pre-2.0 (it's broken in those):
 - More info in this thread
 cd /d D:\Programming\android-sdk-windows
 copy /y platforms\android-1.1\tools\dx.bat platforms\android-1.1\tools\dx.bat.ORIG
 copy /y platforms\android-2.0\tools\dx.bat platforms\android-1.1\tools
 copy /y platforms\android-1.5\tools\dx.bat platforms\android-1.5\tools\dx.bat.ORIG
 copy /y platforms\android-2.0\tools\dx.bat platforms\android-1.5\tools
 copy /y platforms\android-1.6\tools\dx.bat platforms\android-1.6\tools\dx.bat.ORIG
 copy /y platforms\android-2.0\tools\dx.bat platforms\android-1.6\tools

# Install the ZXing codebase (ZXing-1.4.zip):
 - Source: http://code.google.com/p/zxing/downloads/list
 - My root folder: "D:\Test\zxing-1.4"

# Edit "zxing-1.4\build.properties":
 - Note double-backslashes!
 - Update: WTK-home=C:\\WTK2.5.2_01
 - Update: android-home=T:\\Programming\\android-sdk-windows

Now, if all that went well you should be ready to build it!

# Open a "cmd.exe" window to work in.

# Set appropriate environment variables for the tools:
 set JAVA_HOME=C:\Program Files\Java\jdk1.6.0_18
 set path=%PATH%;D:\Programming\android-sdk-windows\tools
 set path=%PATH%;D:\Programming\apache-ant-1.8.0\bin
 set path=%PATH%;%JAVA_HOME%\bin

# Set the environment variable for the root of your project:
 set MY_PROJ_ROOT=D:\Programming\workspace\zxing-1.4

# Running the build (repeat as necessary):
 REM I added this top-level clean step to my process -
 REM It helps when you have tried building from the wrong level!
 cd /d %MY_PROJ_ROOT%  &&  ant clean

 REM Standard build step (see note in android/build.xml)
 cd /d %MY_PROJ_ROOT%\core  &&  ant clean build-no-debug

 REM Debug is the default target - I prefer to name it explicitly
 cd /d %MY_PROJ_ROOT%\android  &&  ant clean debug

 REM OPTIONAL: You can instead build the release version but it's not signed
 cd /d %MY_PROJ_ROOT%\android  &&  ant clean release

# Inspect and use the build output (the APK file is produced if successful):
 - %MY_PROJ_ROOT%\android\bin

At this point you should have successfully built the Barcode Scanner app for Android!

Load it upon your emulator or phone and see how it works. To try it on your phone, you will probably need to uninstall the current version.  Then you will need to check the “Settings -> Applications -> Unknown Sources” box on your phone to install a debug binary like this (As always, be careful when testing in this mode!!)  Copy the APK to your phone and install it.  Voilà!  It worked for me.

Next steps:

• Seeing if I can do this from within Eclipse.  A somewhat outdated Knol suggests this is possible, but I had no luck with it. Done!
• Install a Subversion client to get the latest code and see how that goes (my expertise is with Clearcase and such so this should be interesting). Done!
• Modifying some code! Done!

Disclaimer: I’m NOT attached to the ZXing project at all, I’m just putting this out there in the hopes that it’ll help others have an easier time at this than I did.  I probably cannot help you with any issues you have (especially for other host or target platforms).

Flash Drives and The Removable Bit

Something I stumbled upon today was a Lexar utility that can flip the removable bit on a flash drive.  Disclaimer: this may not work for you or could even cause problems! As you may have noticed a flash drive appears as a “removable disk” in explorer, and (as I understand it) cannot be partitioned in XP or lower (Vista doesn’t seem to care as much from what I can tell).Indeed, it works, making the drive go from “removable” to “basic” (that is, fixed or local).

As I was doing all this in Vista with UAC enabled I was alarmed when it appeared that once I flipped the bit there was no flipping it back!  It’s not necessarily bad but I prefer to keep it removable unless otherwise necessary.  After lots of flailing (and finding a few posts alluding to this one-way trip problem – but no answers) I ran the utility via the “Run as administrator” option. This allowed BootIt to see the “fixed” disks (which the flash drive becomes a part of after flipping the bit).  Be VERY careful not to mess with the wrong drive of course! I think Vista is more protective of the fixed disks, thus the unexpected hurdle.

Note: I was able to flip the removable bit on Lexar and A-DATA drives, but I had no luck with my 4GB OCZ Rally2, a Sandisk Cruzer Micro or with an old Pocket Disk brand 128MB drive (handy for small images) which all stayed resolutely removable.

Read more about the Lexar BootIt (Flip the Removable Media Bit) tool here and here (I used v1.07, which I think is the only one out there).

Adding bootable ISO images to UBCD4Win images

It’s pretty easy to add bootable ISO images to UBCD4Win (this is tested with v3.22 and should work with v3.50 equally well).  Today I had LOTS of “fun” dealing with a lost boot manager on my triple-boot laptop (Vista, Ubuntu, Win7, oh my!) and I snagged copies of GParted and Super Grub to try and save the day.  All I did was download them and add the ISOs to the /images tree on the prepared flash drive.  Then I added a couple of entries in /menu.lst as follows:

title Super Grub
map --mem (hd0,0)//Images/super_grub.iso (hd32)
map --hook
chainloader (hd32)
rootnoverify (hd32)

title GParted
map --mem (hd0,0)//Images/gparted-live.iso (hd32)
map --hook
chainloader (hd32)
rootnoverify (hd32)

The ISO names must match what’s in /images, and I’d avoid putting spaces in the names too.  The usual disclaimer applies: I figured this out by trial and error but it worked fine for me.

Super Grub ended up working and saving the day!  GParted, not so much… I had to put it on a CD to get it to fully work.  This is no fault of GParted… as far as I can tell the same problem would affect ANY partition tool. GParted actually got stuck in a repeating loop of attempting to mount a drive and I think it had to do with how the boot process involved resetting the USB hub (thus resetting the flash drive).  Other partition managers that pass through a USB stage also experienced the same issues.  Booting GParted from the CD-RW appears to avoid all that.  I had the same problem when I attemtped to mount and install the Ubuntu 9.04 ISO from that USB drive today… it worked fine from a DVD.  So, while this method works in theory it may not always be practical given the limitations of booting from USB.

What’s nice is that once I have a small stable of utilities like this I can eventually burn the whole thing onto a DVD for near-universal usage (even on machines that won’t boot from flash).  Pretty handy! 

Update: there are better ways to do this menu thing I’m sure… and apparently the CD ISO uses an entirely different menu system for some reason.  So at a minimum /BCDW/BCDW.INI also needs to be updated (but I’m not yet sure what that should look like).

127.0.0.1       localhost
::1             localhost

I made a backup of the file, then I let Defender “clean” it.  OK… it only removed the 127.0.0.1 line (in red above).  Weird: that’s a pretty standard setting and it doesn’t seem like it should be going anywhere.

I searched around for this and found this thread on the matter.  I then used Windows update to get the latest version of the Defender database (it was last checked about 18 hours ago) and reverted the “fix” Defender had made (in Vista you must edit the hosts file with an editor running in Admin mode… as always be careful!)  Sure enough, it found and installed a newer version and a re-scan of the hosts file showed… no problems whatsoever.  Apparently one of Monday’s Defender definition updates might have had a bug in it.

Note: This is NOT to imply this is always a false alarm!  But if the only line that was removed is the standard localhost address as above, update Defender and re-scan.  This “problem” may not be a problem after all.

Now, I wonder how many people screwed up their hosts file today by letting this rather ubiquitous setting get removed?  I can imagine there are some apps that’ll be unhappy not to find a localhost route.  If this post helped you avoid some fun config headaches later please drop a quick comment.

Update: more info at this site about this issue.

I did some searching… turns out there are lots of complaints out there about this sort of issue, but almost nobody posted the correct way to fix it when Vista SP1 and WMP11 are involved.  Most solutions revolved around WMP10 or earlier.  The problem is, for some bizarre reason you can’t manage your licenses in WMP11 (there is no menu item for it and apparently all backup/restore functionality is gone!)  Why?  I have no idea.  Glad I don’t depend on it for anything else!

I uninstalled and reinstalled Netflix… didn’t help.  Lots of suggestions out there to downgrade WMP11 to 10 or such, but those are fraught with caveats.  Reinstall Vista?  Right!  When my furnace won’t work I don’t repaint the living room and redecorate the bedrooms… why should one rebuild the whole system for an isolated failure?  Besides, apparently that doesn’t help either.

Finally, I stumbled across a useful answer! Close IE and run the following command.  CAVEAT: supposedly it resets ALL your DRM license info… beware!  That wasn’t a concern for me (I’m wary of buying any DRM-wrapped media that could evaporate on a whim) but it could be an issue for you.  The command is:

C:\Program Files\Netflix\Netflix Movie Viewer\ResetDRM.exe

After a couple of warning windows (read them carefully!) it completed.  I reopened IE, clicked play on a Netflix movie and instead of crashing it notified me it needed to update something in WMP (as it had done long ago in XP).  I let it, and after that it works like a charm!

Well, almost… looks like it’ll complain about getting a license every time you use it (because of IE’s “protected mode”?  I dunno but I don’t feel safe to disable it).  It seems to work after a prompt, but it makes me think it’ll eventually fail again.

Was all this work worth it though? We’ll see… I don’t trust that it’ll be reliable and if it wasn’t free as part of my Netflix membership I would be very hesitant to use this at all – especially when such DRM “rights” could suddenly disappear tomorrow or, as we see here, every time you use it.  DRM is nothing but a nightmare and apparently it’s only getting worse.  I’d go to Blu-Ray but I don’t want any “Surprise!  DRM doesn’t like your big new monitor!” problems (also reported, which HDCP compatibility supposedly helps but it’s a “gotcha!” I don’t care to risk right now).

References:

http://www.annoyances.org/exec/forum/winvista/t1201447823

What all does one write on a blog that’s purposely sanitized so it won’t offend past, present or future employers? Something intended to be neutral and unassuming? Frankly not a heck of a lot.

*Sigh*