A shout-out for the “Writing Idiomatic Python” ebook!

I recently purchased the Writing Idiomatic Python bundle by Jeff Knupp (at http://www.jeffknupp.com/writing-idiomatic-python-ebook/). As I continue to refresh and reinforce my Python knowledge I figured it’d be handy, and indeed it turned out to be pretty useful to me. However, I found some typos and small errors, so I submitted them to the author. Lo and behold, my name’s in the contributor acknowledgements in the latest revision! :-) I think that’s the first time that’s happened… and quite cool too!

Note: I paid regular price for the book and am not being compensated for this endorsement.

Share
Posted in Software and Hardware development | Tagged | Leave a comment

Python and Bitwise Rotation

The other day I found myself wondering how to perform bitwise rotations in Python, particular bit rotations (circular shifts). There’s lots of complex ways to do it - bitstring, bitarray, and others – but I wanted a more compact and efficient method.

Turns out it’s relatively easy… the C idiom for bitwise rotation is a good starting point, but I’ve extended it to handle an arbitrary unsigned bit string (one that is first truncated to fit in the width of the virtual “register” we’re shifting in – just so it doesn’t go totally bonkers if it gets an “impossible” value). It’s also robust enough to handle excessive rotations (where a value only n bits long is rotated more than n bits).

Here it is along with some test code:

###########################################################################
# Rotating bits (tested with Python 2.7)

from __future__ import print_function   # PEP 3105

# max bits > 0 == width of the value in bits (e.g., int_16 -> 16)

# Rotate left: 0b1001 --> 0b0011
rol = lambda val, r_bits, max_bits: \
    (val << r_bits%max_bits) & (2**max_bits-1) | \
    ((val & (2**max_bits-1)) >> (max_bits-(r_bits%max_bits)))

# Rotate right: 0b1001 --> 0b1100
ror = lambda val, r_bits, max_bits: \
    ((val & (2**max_bits-1)) >> r_bits%max_bits) | \
    (val << (max_bits-(r_bits%max_bits)) & (2**max_bits-1))

max_bits = 16  # For fun, try 2, 17 or other arbitrary (positive!) values

print()
for i in xrange(0, max_bits*2-1):
    value = 0xC000
    newval = rol(value, i, max_bits)
    print("0x%08x << 0x%02x --> 0x%08x" % (value, i, newval))

print()
for i in xrange(0, max_bits*2-1):
    value = 0x0003
    newval = ror(value, i, max_bits)
    print("0x%08x >> 0x%02x --> 0x%08x" % (value, i, newval))

###########################################################################
Share
Posted in Software and Hardware development | Tagged | Leave a comment

Trying out my new Colorduino v2.0 – first impressions and tips

After my recent experiments with the Logos RGB backpack I stumbled upon the Colorduino (a cousin if not a descendant of the Rainbowduino), which is a very interesting little device! It not only controls the individual colors, but can vary their brightness (something the Logos backpack doesn’t offer).

Continue reading

Share
Posted in Electronics, Software and Hardware development | Tagged , , , , | Leave a comment

Howto: Getting started with Python development in Windows with MSVS/GCC, Cython, etc.

Note: this is directed at Python 2.7 developers, but the same principles should apply to Python 3.x as well.

First, a shout-out for the Unofficial Windows Binaries for Python site: as noted below you can get the binaries for many commonly used packages simply by downloading and installing them, with no need to compile.

However, when it comes to things like Cython that NEED a working compiler at some point you’ll have to set one up. I wrote this post to help you with that, hopefully saving you many hours of frustration!

Conclusions first: properly configuring Windows to compile for Python/Cython takes a little effort but the hardest part is getting the details in one place (which this post is intended to do). The total setup time if you go the full Windows SDK/Microsoft Visual Studio Express/MinGW route isn’t that long – less than 30 minutes with a decent connection on a fast machine, with the majority of it spent in the download/install phases.

I DO suggest you install Microsoft Visual Studio Express 2008 and the Windows SDK. As for MinGW, I strongly suspect there are limits to what you can ultimately do with it in this context but it’s presented here as an option for the adventurous (and it’s rather nice to have GCC handy in general!)

Continue reading

Share
Posted in Software and Hardware development | Tagged , , , , | 1 Comment

Learning by Hacking: Modding the Logos RGB LED Matrix Backpack

As mentioned in a previous post, I recently bought an RGB LED Matrix “backpack” (with 8×8 RGB LED display) created by Logos Electromechanical. It’s an open design, and you can find both the hardware design/schematic and firmware on that page. And again, kudos for this: it’s one thing to fabricate and ship a product, but it’s makers like you that give the community the opportunity to turn what would otherwise be a single-purpose device into a learning and extending opportunity!)

Continue reading

Share
Posted in Electronics, Software and Hardware development | Tagged , , , , , , | Leave a comment

On the proper use of the I2C bus in general and with the Logos RGB Matrix Backpack

Note: In the course of all this I created an improved Arduino I2C bus scanner. Check it out!

First, a thank-you to Pierce from Logos Electromechanical who clued me in on what I was doing wrong with the backpack.

It turns out there are two common ways to detect an I2C device: address it and see if it ACKs (quick mode), or address it and try to read a byte from it (read mode). Some devices that aren’t set up to write don’t like to be read from and will cause problems if you try to do that (in this case, SDA gets pulled low and stays low!) I thought it was a bug but no, it’s actually how this device is supposed to work.

I selected an address for my backpack of 0×58. It turns out that i2cdetect has some intelligence when it comes to its default behavior – it will decide based on the address how best to perform its test of that address. If the device address is in the ranges {0×30:0×37,0×50:0x5f}, it will use read mode. Outside those ranges, it will use quick mode. See the i2cdetect.c code for how this was implemented in Linux.

So I chose… poorly – at least when it came to the address for the backpack (and keep in mind that the default address of this device is 0x7F, which is invalid for I2C, so you MUST select a valid address via solder bridging the appropriate pads!) Since the high (0×40) bit is already “1″ on this device, you can choose an address within the valid range of 0×40:0×77… I suggest something outside the 0×50:0x5F range though, to avoid accidentally triggering this (otherwise, be careful to use “-q” with i2cdetect). You can quickly tell when you screw up and try to read from this device: everything else will drop from your I2C bus and detect will act like every I2C address is in use by a device! (Unplugging the backpack from the circuit and from its power source for a moment will reset its state.)

You can read more about the I2C protocol here and here, learn about special addressing modes here, and see the specs here.

When using the Raspberry Pi model B note the I2C bus on the main header is numbered 1 (older boards have bus 0 on that header).

Share
Posted in Electronics, Software and Hardware development | Tagged , , | Leave a comment

Ah, Maker Faire! BeagleBone, HexBright, etc.

I had a great time at Maker Faire this weekend! I picked up a few items which I’ll quickly summarize here:

First, I snagged a BeagleBone Black kit. I haven’t had a chance to open it up yet but it looks promising, though I wonder just how much trouble it’s going to be… it sounds like it will be more daunting than my Raspberry Pi (which I’m finally making good use of!)

I also picked up a Logos Electromechanical Labs RGB LED Matrix Backpack to fiddle with. I’ve been busy getting I2C devices working with my Rpi and thought this would be a fun addition! (I’ve previously fiddled with them on the Arduino, but the Wire library on Arduino leaves much to be desired in terms of correctness). Sadly, while it works with the Arduino, the backpack appears to hose to I2C bus on the Rpi (note that since this backpack is a 5V device I am using an appropriate level shifter to interface it to the 3.3V Rpi). It seems that once it’s addressed (or even detected, e.g., via i2cdetect) it doesn’t correctly relinquish the bus until the backpack is disconnected and powered off. I’ve emailed them to see if they know why this is and how to fix it (I suspect it’s a firmware problem on the backpack, which is AVR-based… but I’m not sure how I’d update it yet).

UPDATE: the backpack works fine with the Pi – it was the nut behind the wheel that was the problem. More in an upcoming post. :-D

My most unique purchase was the HexBright FLEX 500 lumen Arduino-powered LED flashlight! :-D I was able to reprogram it pretty easily, though it doesn’t work with the Arduino IDE 1.5.x yet (also, when adding this target, merging their boards.txt to your boards.txt is the easy way to go… it didn’t work for me otherwise).

Note that I’m not thrilled with the charger that comes with the HexBright… I recently learned a LOT about off-brand USB chargers I was unaware of, particularly with regard to safety, from this EEVBlog post among other places, and that makes me wary of chargers that look like knockoffs. While I’m happy to use cheaper parts when possible (I’m certainly not a hardware purist!), I expect the things I plug into my wall to have a a valid UL stamp on them at the very least. Bonus points if they spell the words right on it (“inter-mation equipment”? Looks like copy-pasta from a few other questionable chargers I’ve seen teardowns of – probably the same maker slaps whatever name you want on them). And no, that serial number is not “serial” at all on cheap chargers.

20130519_181233

Again, not knocking the flashlight, which seems pretty solid… just not thrilled with the cheap USB plug-in charger it comes with. Now that I know more about what goes into these small form-factor chargers I can appreciate that you don’t want to skimp when it comes to quality on them.

Share
Posted in Electronics | Tagged , , , , | Leave a comment

Windows and the Case of the Unexpectedly Incorrect Desktop Icon

Tonight I moved my installation of Eclipse to another folder. I made sure to update the registry accordingly… this is where NirSoft’s free RegScanner utility came in handy: it’s something like Agent Ransack (another awesome free utility) for registries. I found all the instances of the old path, exported the .reg file, updated it, then imported it. It worked, and saved lots of tedium hunting through the registry manually for the same things.

Then, in the course of updating things I updated the software for my OWON oscilloscope. Up to this point everything worked fine, but when I put the ‘scopes icon on the desktop it was the Eclipse icon, not the one that is very clearly the one for the scope.

Continue reading

Share
Posted in Windows | Tagged , , , | Leave a comment

Github: long-time user, first-time contributor!

I’ve finally posted some hobby code to Github! <

In this case, an Arduino project I’ve been dabbling with for a few weeks now… I’ve successfully combined a Wii nunchuck controller and a long RGB LED strip (WS2811-based) using an Arduino Mega. It’s very much a work-in-progress, but it’s the start of building a little body of work out there.

 

Share
Posted in Software and Hardware development | Leave a comment

Memristor simulation with LTspice – a practical example!

Lately I’ve been getting back into the EE groove with some random electronics projects (such as an Arduino-based LED strip driver). I’ve also been dabbling with different circuit simulators like Micro-Cap (which I used back in college but which is VERY expensive for the regular version… still, it’s handy for quick little sims to learn new things, especially in the digital realm).

Today I stumbled on LTspice from Linear Technology, which I found to be surprisingly intuitive (and given that it’s based on the venerable and powerful SPICE it’s that much more worth learning). It’s not as animation-friendly as Micro-Cap but it’s pretty easy to make custom components (and where else can you simulate the actual inner workings of a 555 timer or a 741 op-amp?) (And yes, the 741 is not something you’d reach for normally nowadays, but they are instructive nonetheless.)

Memristors have intrigued me since they made the news a few years back, when discoveries were made of practical memristive materials. You can read more about them on Wikipedia and so on, and though very subject is not without controversy I personally think they will prove to be a valuable technology going forward. I found them intriguing from both the EE and the MSE perspectives.

Side-note: if you find yourself wondering more about why a device does what it does (in my case, the humble MOSFET) than regurgitating Kirchoff’s Law and applying Thévenin’s theorem, you’re probably better off in Materials Science Engineering than Electrical Engineering. At least that’s what happened to me… that and a very grumpy EE prof who made it clear we’d all be obsolete, unemployed and grumpy like him soon anyway. And you can always fall back on CS when the semiconductor market tanks…

Given all that, today I decided to see if I could simulate a circuit with a memristor in it. Problem is, while some sims have memristors, LTspice does not, and apparently nobody out there has published a simple example that actually works (I found videos – who writes a text tutorial in a video? – and a broken webpage, but I never found a working sim). So, I made one… and after a few false starts I got it to work. Not bad for an evening at home!

To try my demo out, simply click here to download my LTspice-based memristor simulation and follow the instructions in the README.txt file for how to install and use it.

And to reiterate, this is a demo based on the SPICE model of a memristor as described in one paper… this is for educational purposes only and no, I can’t help you with your homework. :-D

Share
Posted in Electronics | 1 Comment

Exporting passwords from multiple Chrome profiles (to LastPass, ChromePass, et. al.)

Update 1 at the end.

Note: this post applies to multiple utilities with this use case, such as NirSoft’s ChromePass and LastPass. This also applies only to Windows (though it may have similarities to workarounds for other OSes). Finally, BEWARE: this involves carefully fiddling with some folder names. There be dragons!

So, I recently decided to look at better password solutions than what the various browsers afford. I looked at KeePass, RoboForm and LastPass. Your mileage may vary, but for now I’m trying out LastPass.

In the course of this I stumbled on the fact that it’s surprisingly cumbersome to import passwords from Chrome (a browser I’ve come to like but which is at times cryptic). I ended up  turning to NirSoft’s ChromePass in an attempt to get the passwords exported from Chrome for later import. That failed, in the same way LastPass failed: it was happy to import one set of passwords only, no matter which Chrome profile was active. Note that the Windows login is the SAME in each case, thus there are multiple Chrome profiles visible at once.

Turns out, the only profile ChromePass, LastPass and others ever refer to for password data is:
C:\Users\%USERNAME%\AppData\Local\Google\Chrome\User Data\Default

If you have multiple profiles you’ve used under this Windows login, you will have folders such as:
C:\Users\%USERNAME%\AppData\Local\Google\Chrome\User Data\Default
C:\Users\%USERNAME%\AppData\Local\Google\Chrome\User Data\Profile 1
C:\Users\%USERNAME%\AppData\Local\Google\Chrome\User Data\Profile 2
etc.

However, if you’ve removed and re-added profiles you may not even HAVE a profile folder called “Default” here! (Resulting in an empty password list when you attempt to use LastPass or ChromePass to get to them.)

Thus, if your current Chrome login happens to be in that profile folder, you’ll get the info from that, but if you have another Chrome login (or for other reasons no longer have the Default profile folder) you’d be outta luck – until now.

What to do? It’s surprisingly simple! If what you see when importing or exporting passwords is NOT what you expect to see (that is, you are seeing the Default profile only), do this:

  • Close Chrome (clear your cache first – always a good idea)
  • Temporarily rename the profile of interest to “Default” by renaming its folder accordingly (if Default already exists, rename it to something else temporarily before doing this). You may exercise some trial and error for multiple profiles.
  • Now, do what you need to do – whatever profile is in a folder called “Default” is the one that’ll open up in your utility/plugin. You can open Chrome and import passwords to LastPass, leave Chrome closed and use ChromePass to dump your passwords, etc.
  • Done? Close Chrome!
  • Rename the folders back to the way they were

Again, BE CAREFUL! Would that it be simpler, but at least it works. Hope you find this helpful! :-)

PS: When importing passwords to LastPass from Firefox or Chrome do be careful: I’ve already run into cases where not all passwords get imported. It’s a bit of a pain, but it should be worth the benefit of being better organized.

UPDATE 1: While this worked for ChromePass, LastPass (as a browser extension) can be a little more tricky: after renaming the Default profile (if necessary), COPY the profile of interest to a folder named ”Default”. THEN open Chrome, do your import to LastPass, close Chrome, and remove the copy you made. 

Share
Posted in General | Leave a comment

Elementary! MySQL did it, in the CPU, with the the leap second!

I awoke to an alert from a small Ubuntu 12.04 server I have on Linode: lots of CPU usage this morning,which is odd since I haven’t logged in for a while. Hmm. Looked at the CPU graph first, and the high usage was all since exactly midnight. Hmm again.

So, I groggily log in and start looking around, and immediately see that MySQL is using LOTS of CPU all by its lonesome. I immediately wondered if somehow a hacker got in or something (given the recently reported MySQL bug) so I went to look for some hints on investigating that. On a whim (remembering the odd timing of this) I searched Google for “mysqld went to 90% cpu at midnight”. Sure enough, it’s probably a bug.

Turns out, this weekend sports a rare “leap second“… and that triggers a kernel bug which in turn causes MySQL to start eating CPU like a bad gambler at a casino buffet.

I found the following suggestion (found here and here) worked well enough for me on its own (fingers crossed anyway):

sudo date -s "`date`"  # <-- Mind the backticks!

I ran that and the problem immediately disappeared. I didn’t need to reboot or restart mysqld (rebooting would also fix it apparently, but restarting mysqld would not).

Hopefully this bit of advice serves you as well!

Share
Posted in Uncategorized | 2 Comments

Android, Eclipse, XML files and weird error messages

More than once, I’ve gotten the following weird problem when looking at an XML layout for Android in Eclipse:

Failed to parse file C:\Eclipse_workspace\ButtonTest2\res\drawable\arrow_1.xml
Exception details are logged in Window > Show View > Error Log

The error itself isn’t exactly informative, and certainly doesn’t make sense:

org.xmlpull.v1.XmlPullParserException: PI must not start with xml (position:unknown @1:5 in java.io.InputStreamReader@1c167686)
at org.kxml2.io.KXmlParser.exception(Unknown Source)
…….

Hmm. XML files should start with an appropriate declaration line, yet this says my XML file shouldn’t have it. That’s… weird. Normally I just remove the line til it stops complaining but as I’m playing with the second pre-release of r17 android SDK (some key components have been broken lately) and feeling bold I decided to understand this once and for all. What follows is the text of my answer on StackOverflow earlier tonight:


I’ve beaten my head against this wall a few times and now I have the answer (I *hate* mysterious behavior, especially when it seems to be random).

I just ran a 4-way test (Unix and DOS, with and without the binary “EF BB BF” BOM header/signature). I found that Eclipse does not put a binary BOM header into XML files it creates, period. If you add a file that has the `”<?xml … ?>”` declaration header, it works correctly ONLY if that binary signature is ABSENT. (Whether EOLs are Unix or DOS doesn’t matter). If you have *both* forms of header you get this problem. You can have one, the other, or neither, but never both.

So, if you create a new XML file in Eclipse you can expect (in 3.6 – 3.7.1 anyway) that it has no binary BOM header and you should properly put a `”<?xml … ?>”` declaration header in there to identify it as Unicode. If you IMPORT a project that has files with binary BOM header signatures or use an external editor that likes to add them, that’s where all this fun begins.

Ideally, ensure XML files with declaration don’t have binary BOM header before importing them… otherwise, remove that `”<?xml … ?>”` declaration header whenever Eclipse complains… short of removing the binary header that’s all you can readily do. (It’d be nice if Eclipse handled this more consistently… or at least gave you a way to identify and fix the problem readily by removing the binary BOM header…)

If in doubt, examine the file with an external hex editor, preferably one that is NOT Unicode-aware (WinHex worked for me, YMMV). Some text/hex editors will auto-detect Unicode and may auto-add a header when the file didn’t really have one.


Postscript: I use IDM’s UltraEdit / UltraCompare primarily. They work great but they’re a little too good at determining if a file is Unicode or not, to the point where these four text files ALL had the binary BOM header in hex mode!! WinHex showed the exact situation, and indeed only two of the files actually had that header. However, I’d like my regular editor to work correctly. Time to write IDM an email…

Share
Posted in Uncategorized | 1 Comment

Thoughts on mobile app crash detection and recovery

As I work on updates to my app, I’m happy to see it’s working quite well. However, inevitably there will be some boundary case that causes an exception, and as I add features (and more importantly, persistent settings) an addition risk comes into play: failed restarts after a crash.

Continue reading

Share
Posted in Android | Tagged , , , | Leave a comment