Postcards from the Bleeding Edge

SOPA is bad news

2012-01-18T09:44:00.001-08:00

I haven't been paying a whole lot of attention to American politics of late. At least one variant of SOPA, as proposed, will mess with DNSSEC.

I believe - wholeheartedly - that DNSSEC has the ability to improve the security of the Internet for everyone. It's a critical component of my cerowrt project, comcast just rolled it out nationwide, many other providers are also doing so, and that has taken tens of thousands of people, over 10 years of effort to accomplish.

I have also been working for years to make DNSSEC just work, only to look up now, to see clueless politicians in the pay of a a few lobbyists playing with technologies they don't understand.

I've been busy this year, on another front - trying - and succeeding!! - making the Internet, actually better, for everyone.

It really bugs me to see all the time, energy, and money, that can go into screwing up the internet, especially vs all the time, energy, and money that goes into making it better.

I don't have a comprehensive grip on what sopa does, but here's one reference that seems good. Doc Searls also ranted well...

Seeing wikipedia go dark today, was a terrible, terrible thing. I'd like to keep the light of knowledge, burning bright, throughout the world, for everyone, all the time.

Departing France for England, then 'home'.

2012-01-16T08:53:00.001-08:00

So, I looked up, and realized that it was time to leave Paris.

Tomorrow.

I'm sad to be packing up and leaving, and mad at myself on a few particulars, but on the whole, it's been a wonderful experience to have spent time in one of the still beating hearts of civilization.

As a tourist I got to see the Eiffel tower, and tour the entire Louvre - (it wasn't the Mona Lisa that grabbed me, it was the painting on the opposite side of the room that really blew my mind - and the statuary - oh, my!)

I ate dinner at nearly every restaurant between my apartment and my graciously donated office at LINCs lab, and a few other places. I got turned onto some wonderful places with friends that I will never find again. The wine, was wonderful... I had a couple dates, too, but romance escaped me - and I was working my brain dry... I spent 99.9 % of my time obsessively trying to beat the bloat.

I met some people I'd only corresponded with for years, helped a couple students, gave a couple talks, did a ton of research, and my life revolved around work, primarily. Darn it. Again.

There was so much more I could have done - in particular I'm sad I missed seeing both the catacombs and Monet's gardens, but I hope to be back one day, and those will be tops on my list. I wish also - although I fell in love with Duc Lombards and the two other big jazz clubs in Halles - that I'd actually got to *all* the jazz clubs, and seen some plays and concerts, but it was very hard to navigate alone...

I didn't manage to learn that much French. I will have to study the language far more fully before I choose to return. I did learn lua, though.

I'm off to London now for a week (I'm giving a talk at UKnof21), and I hope to spend time there, seeing things and people that I've always wanted to see, in that country. Then... Back to Florida to see my folks, and after that...

I have NO idea. Usually I write up a summary of the last year, and make some new years resolutions, and review the old, and I guess I'll start doing some of that, while trying to ingest as much of Europe as I can, in my time remaining here.

I wish I'd got to Amsterdam, Spain, and Italy on this trip, too.

Ahh, well. The work was good.

An incredible stream of co-incidences passing me by

2011-11-07T05:46:00.000-08:00

For the past couple days, while waiting for code to compile, I've been playing an old song of mine, "Living in the ooze", revising the lyrics, coming up with a bridge (finally! 6 years without a bridge!) that I think might work. This morning I even went out to get batteries for my recorder. I didn't get them because I don't know how to pronounce 'batteries' in French, and staggered back, defeated, to my apartment in Paris.

I've told the story behind the song in multiple posts over the past 6 years, and we're coming up on asteroid appreciation day feb 29th, next year, and I don't have a plan or place for the party... and in part those stories are always about impossible co-incidences and weird stuff happening at all the same time...

I have NOT been paying attention to space stuff AT ALL for the last 9 months, being deeply immersed in bufferbloat. I even stopped reading the Arocket mailing list. I spent the weekend completely immersed in a set of algorithms that finally, finally might reduce latency over wireless to sane values once again, only to end up with one bug per line of code and multiple kernels that wouldn't boot. Never left the house all weekend. Still haven't made it to the louvre. Or anywhere else in paris but the LINCs lab. I'm insanely frustrated with this project.This morning at 12:40 AM I was ready to chuck it all and get on the next plane to anywhere that didn't have computers... but was stuck on thinking where on earth that might be, so I did a few other things about my stuck-ed-ness..

I haven't been reading the news, either, as news.google.com insists on coming up in French no matter what I do. I'm still stuck on the six different ways to pronounce 'e', in my French-english dictionary.

So, later this morning, in an impossible co-incidence, I meet someone who just spent the summer in Nicaragua, who points me at her huffpost article, and I send a link back to one of my adventures in australia which happened to be the last thing I'd written about this particular song... and happens to be about some weird coincidences in spite of my agnostic-ism...

And then... I notice... at the top of HER post...

Quarter mile wide asteroid to miss earth Tuesday.

Nerts. It's going to miss Texas. It almost hits the moon though! That would have been spectacular! And the article has quotes from Lance Benner who I've exchanged multiple emails with over the years, and I still point actively at the data he collects on plausible rendezvous trajectories...

Perhaps the universe is telling me something. About what, I don't know. Perhaps some sort of sacrifice to it is needed. Perhaps it - or something in it - needs attention.

Either I get back to recording this song or go back to work, or go for a rendezvous.

OK, I've googled for how to pronounce batteries. And I wonder what can rhyme with UV55?
PS: Boycott the Hôtel Ibis Sainte Catherine, Bruxelles

12 suggestions for startups

2011-10-15T05:17:00.000-07:00

I'd written this months ago and meant to post it then. Having been through 14 startups now (of which 3 I started, 1 (none that I started) that worked out ok), I figure I have sufficient background to give advice... especially to myself. I tend to forget one or another of these points much to my own detriment, in the heat of creating a startup.

I keep forgetting to apply points 10 & 11 below to my own work. At present, I'm merely applying startup-like methods, some of which are working, some not, to a major R&D project. The number of points missing on the strategy are sometimes glaring - I merely want to fix a problem (bufferbloat) so well that it never bothers anybody again. As for profit motive, I have none - merely cutting my own annoyance level regarding the behavior of wireless networks back to "mellow", almost suffices.

12 suggestions for startups

1) Retain control. Most successful startups go through three rounds of
funding.

If done wrong, by the last round the founder(s) generally reduced to a tiny minority shareholding and have lost control of the company... and are usually forced out by the 4th round or IPO.

If done right, the founders retain 80% or more of the stock in the 3rd round.

I note that 9/10th of startups are not successful, and that a large percentage of the ones that 'make it' did so without benefiting the principals or employees to any real extent, only the VCs.

2) Paul graham is good. I'm a big fan. The ycombinator model appears to be working, but as to how much it meets goal 1, above, I currently have no insight.

3) Have a pitch, and a plan, pitch it often, *listen* to the responses and objections, and revise after every meeting. Don't be deterred by failure. Make sure your goals are shared with the people working with you, as well.

4) Selling something is great.

Selling something you can actually make and sell at a profit is vastly to be preferred.

5) Get to plausible promise before seeking any money at all. If you can't do a startup's initial development on what you already have, you can't afford to take risks of this size, and should stay gainfully employed at something else.

Once you start chasing money, chase it hard, chase it continously, get it in the bank, and spend it appropriately. While there is such a thing as 'too much money' while in growth mode, it's a nice problem to have.

6) Decide on your exit strategy early. "go public, "stay private", "make money for 5 years", "lose money for 5 years".

Cynically I note that you don't have to share your exit strategy with your employees or your investors.

It does help in planning and in motivating your people if you know what your exit strategy is - there are very different motivators for 'GOING IPO IN 2 YEARS' vs 'Make money, reliably, starting in 2 years', or 'provide a valuable service', or 'solve a global problem', and it changes the kind of people you get.

7) Incorporate early. Twice. The first company loses money, the second is in reserve. Both are handy to have around. Shell companies in particular establish credit that exists independent of your own income, over time.

It is best to have had a company around for 3+ years before starting to really use it.

If it were legal, given todays legal and tax environment, I'd incorporate kids at birth. As it is, at the moment I'd recommend incorporating 'em as soon as legal in Delaware rather than overseas. It really complicates you life unnecessarily. (that said, doing business with other businesses outside of the US is mildly easier, and shipping your kids overseas will give them a bigger picture than they can get in the US of world needs)

8) Get a good lawyer. Also, get a good accountant. Take the advice of both particularly as regards to points 5 and 6, not me! Two lawyers and two accountants are an even better idea. Bad lawyers and bad accountants have sunk more than a few startups.

Have a clear goal, corporate rules, etc, laid out, in a mission statement, etc.

I have no opinions regarding sub-s or LLCs. A few years back LLCs were all the rage. Talk to two lawyers and one accountant about it.

Regardless, you're going to lose money for a while. It's good to make that tax-deductable any way you can. Cash-flow will always be a problem, whether it's early days of no income or while facing unexpected growth.

9) Contract to hire. Never hire until you have to. The costs both of hiring someone and of firing them far outweigh the extra costs of paying contractors of various worthiness. Getting business insurance is a problem, getting health insurance is also a problem.

Limit your fixed commitments rigorously.

Contract yourself if you want. I use MBO partners for this - they get me a pair of 401ks, 1m in business insurance and take care of a ton of details that I don't want to, in exchange for 5% off the top of my billings (in the US) - where most agencies take 40%.

10) Recognize your own faults. If you are a detail person, get someone that can stay focused on the big picture. If you are a big picture person, get people that do details. If you can do both, do both, but on separate days. REMEMBER to do both, regularly.

(Being a detail person myself, I have to reset with long weekends periodically to review the mis-fired plans and replan. I've had to do a lot more of that this year, than I'd like)

In all cases, having a good and well enabled AA/secretary working for the CEO primarily is tremendously useful. Get stuff WRITTEN DOWN. (I use transcribr.com whenever possible)

Also having a good shared scheduling system is helpful. I could go on for pages here...

11) Set goals, and plan, rigorously, and both conservatively and optimistically. Revise your plan monthly. Software developers are notorious for over-estimating what can be done in a month, and underestimating what can be done in 2 years. Marketing guys are notorious for missing trends until they've already happened, and selling visions of things that can't be built by any software developer on the planet and promising them 'tomorrow'.

12) If you are risk-adverse, don't do a startup. By all means, DO! form a company to fund your own interests, and lose money with it profligately, it makes the IRS mad, and that's worth it in itself.

The seed capital to accomplish the starting procedures above is about $15k. Yearly, maintaining the corporations depends on your locality, but runs less than $1k each, accountant < $1k, lawyer less than $3k, and if you can't find a way to lose 3x that much money pre-tax on a regular basis to make up for it, see multiple points above.

Steve Jobs, RIP.

2011-10-06T15:39:00.000-07:00

I am going to miss Steve Jobs. I'd admired/hated/loved/wanted to be like/unlike him for much of my adult life, and now that he's gone, I can't think of anyone else in the industry that could have stirred up such emotions and thoughts in me. He spoke well, here at Stanford.RIP, Steve. I intend to stay both hungry and foolish, for as long as I can, thanks to your inspiration.

Welcoming my father to the blogosphere

2011-09-18T10:22:00.000-07:00

Ron Taht, my dad, is finally, blogging! My mom and I have been encouraging him for years to stop wasting his time with letters to the editor, which would - whenever published - always get published, truncated, with salient points removed.

A few months back I showed him how to use blogspot and finally - after a few false starts, he started generating some good stuff, at lengths more suitable to what he has to say.

I'd delighted to see him finally getting his full say - and while my dad and I don't see eye to eye on many things, he instilled in me a great love of debate, that I didn't exactly appreciate when I was younger, while I seeking moral guidance rather than debate. Back then, he'd always pick the opposite side of whatever I was thinking about, no matter what he actually thought, just to sharpen my wits (and leave me confused about, well, just about everything)

Now, after finishing his career as a prosecutor, lawyer, and judge, he speaks with conviction, about what he really thinks, and *I'm* the one that automatically picks the opposing side when discussing anything with him!

I'm very glad he's finally writing his stuff down, and speaking his true thoughts from his heart and mind.

He has tons of entertaining stories that I'd like him to blog about, too - fishing tales, golf stories, multiple episodes in court both tragic and funny, hysterical funny scenes from housing sales, and scary stuff from his prosecutor days...

But, as he's not much of a typist (as yet), he's primary writing about the issues that concern him now, deeply, about the current problems and future of America... and I find myself agreeing with him far more often than I'd like (or am willing to admit, while debating with him)

I hope for a big welcome from the blogosphere it's newest 75 year old member at ronsravings.blogspot.com, my dad, Ron Täht. Comments and criticism of what he's writing about will be deeply appreciated.

Any other musicians at linuxcon in Vancouver?

2011-08-17T09:31:00.000-07:00

I was wondering if the old band could get together for the 20th anniversary... I've got the management behind borrowing the hyatt bar and the piano....

Me, 46, Cerowrt - RC5

2011-08-12T10:33:00.000-07:00

Usually I write a long, contemplative blog post on my b-day, but I simply haven't had time to gaze into my navel all that much.

I AM however, hoping that the RC5 candidate for CeroWrt proves out to be a good one, and begins to address the mistakes I made a decade ago, and mistakes everyone has been making since the 802.11n deployment.

I spent the last week with the incredibly helpful folk at ISC getting a lab put together to test this release of CeroWrt, and it's looking really good... but I did make the go/no go decision on a RC at 2:38 AM this morning, which worries me - but hopefully there will be few problems.

I'm hoping actually, for a dramatic difference in wireless network behavior for those giving the RC5 a test - certainly in testing I saw some of the cleanest TCP/IP streams I've seen all year.

I also look forward to people exploring all the new ideas inside of Cerowrt - DNSSEC, mesh networking, a local web server, etc, etc.

There are still a ton of bugs left to fix, but no priority ones, and that... is good enough to take a day off on, and enjoy wandering a park or two in california, and play some guitar. I'm off to Vancouver for a pair of conferences next - and I hope to stage a reunion from the band we played in, in Nicaragua - with the flautista, Angel, as well. Got a few new songs in my stack now, notably 'Please come to boston'.

So while blogging has been light, if I'm lucky, things will slow down enough for me to talk about what we've been up to for the last 8 months, and where we're going. Where I'm going next, after Canada... is Paris!

The greatest gift I've ever got for a birthday! - was the help of hundreds of people, all helping to fix, and finish the internet.

I'm in awe and delight. Thank you all, above!

Despite all that sentiment... First up this morning, on my list, is laundry.

Battling the bloat

2011-05-28T07:50:00.000-07:00

So, I've been so busy for the past few months as to have let multiple things slide. I've been all over the US - florida, Georgia, Pennsylvania, NJ, Boston, California, Georgia, and now, I am back in NJ, living out of a suitcase with increasingly irrelevant clothing for the weather.

I hope to jaunt to Europe next, after my billings catch up with me. Then back to Nicaragua.

On the minus side:

My main email went down for a month. I hardly noticed.
I completely forgot an important anniversary... for over a week. The lady involved is not speaking to me.

My laptop crashed 5 days after I'd used the USB stick I use for backups for something else.
I haven't checked my voicemail in a month, at least.

I got no exercise.

On the plus side:

Eric Raymond now has IPv6. So does Evan Hunt. Two down, several billion to go.

The latest and greatest bind9 - with dnssec support - is now available for openwrt in the Cerowrt git repository, for testing, as part of ISCWRT. DNSSEC is a mere 3 configuration commands away!

I gave a fairly well received talk at asilomar about the problems with the Net outside the USA.

I attended jim gettys' recent google tech talk, and breathed the same air as Vint Cerf.

The debloat-testing kernel now has most of the features we were trying to test (SFB,etc) , and it has been updated to 2.6.39.

There is now an extensively debloated (but not perfectly by a long shot) version of
openwrt - which also contains the critical stuff from debloat-testing, so we can test
end to end connectivity in all sorts of ways.

That release is entering the final stages of testing, and is codenamed, Capetown. It works on the netgear wndr3700v2, which is a wonderful piece of hardware.

And some routers using that are now up and running, capetown, South Africa, as part of the Bismark project, which I'll be helping out at through mid-august.

The last 4 weeks of my life I worked at a level I have not worked at since my late 20s.

I'm beat. And I'm taking the weekend off. Hopefully someone else will enjoy this stuff and put it to good use in their research into battling the bufferbloat problem.

Operator overload & nuclear troubles at Fukushima 1

2011-03-16T09:39:00.001-07:00

I'd actually written my last blog post on monday, thinking it was tuesday, and wednesday in Japan. Shows how much sleep I've been getting.

The quality of media coverage has improved significantly, but the scope of the cascading failures at the Fukushima nuclear plants has grown - with secondary damage from the explosions and over 500 aftershocks in Japan complicating matters.

The original links I pointed to in my first post have been updated and revised, with MIT's department of nuclear energy doing a better job of filtering through the events involved than any other media organization.

MIT has not addressed the questions raised about the Mark I containment facility that concern me the greatest. I daren't speculate.

Several other aspects of the news coverage and analysis bother me:

A) lack of understanding of the effects of all those aftershocks, and for that matter, coverage seems to be limited to talking about the first quake, even on wikipedia. It's obvious that these had effects, in part, making post-quake inspection difficult.

B) lack of understanding that these were some of the oldest and most obsolete plants in the world, not just Japan. I keep seeing calls for increased safety, or damning nuclear plants for their lack thereof, when these were generation II plants, kept running long beyond their initial design life, due, in part, to the difficulty in getting new plants built.

Generation III+ plants such as the AP1000 have, for example, entirely passive cooling systems, and have safety ratings 1000+ times better than the Gen II plants did. Furthermore they use their fuel more efficently with less waste. Nuclear energy is much more well understood now, nearly 50 years after these plants were designed.

C) The on-going storage of the fuel rods - due to being unable to find another place to store them - is likely the largest danger now, as it appears as though at least one storage pool was damaged in one of the quakes and explosions.

Everybody - on all sides of the nuclear debate - agrees that continued storage of the fuel rods at the facilities was dangerous - and most facilities were not designed with long term storage in mind.

Now, that after the fact, the "out of sight, out of mind" nature of the ongoing storage of fuel rods in ad-hoc facilities in the presence of such debate has been exposed - perhaps some rational decisions about what to do with the spent fuel will emerge.

I doubt the US will become rational on this point anytime soon. I suspect Japan will become so. I also think we'll see a surge of interest in solar power worldwide.

D) with less than 50 operators on duty, that translates out to 12 or less operators managing the 4 reactors in trouble - I imagine that some of the additional failures since monday were in part caused by exaustion and overload, and the inattention to other pre-emergencies due to these factors. It's unclear how many people are monitoring plants 5 & 6.

One of the big causes of both prior major nuclear accidents was operator overload. Too many things beeping, and buzzing and alarms going off, and too much complexity in the control systems.

Future plants - if they are ever built - should have a good offsite management and monitoring facility inconceivable to those in the pre-computer design era.

For all that, I do wonder that the world-wide reaction is overblown. I can't imagine, were I in charge - not sending in every available qualified volunteer and resource available.

For all the worry reported in the press, this (via wikipedia as of about 10AM Wed MDT), remains true:

“To date, the radiation leaks beyond the plant's boundaries have not reached a level high enough to constitute any significant detriment to public health. However, there is still significant risk that a leak at levels high enough to affect public health may occur.”

E) There have been more than a few hair brained schemes floated to cool the reactor pools - for example, cooling the overheating fuel pools by dumping water via helicopter. Dumping water by helicopter cannot be done gently, and would release a great deal of radiation to the crews dropping it.

There is a huge amount of mis-information on the web regarding the deaths of the pilots that flew over Chernobyl, I've been unable to determine the truth of matters.

I find myself tearing up at the dedication of those working to stop an even worse nightmare not just at Fukushima, but throughout the country and the world. I wish I could help.

I also remain in awe and admiration at an high-tech engineering culture and country that could go through a disaster this size, and have under 20,000 dead.

bad wednesday for nukes in japan...

2011-03-14T23:19:00.000-07:00

after my last blog entry, lots of bad, scary information came to light, 2 more reactors ended up with more problems than the first two.

Notably, the presence of spent fuel rods onsite and old flaws in the mark 1 containment system led to my greater concern, particularly after more hydrogen explosions damaged the surrounding area.

According to wikipedia, after a fire at reactor 4, hourly radiation reached 100 000 μSv. That's a big number. A scary, bad, number. But not a (rapidly) deadly number. Reactor unit 3 reached 400,000 μSv. Why people are reporting micro (10^-6)rather than milla (10^-3) bothers me,

Years ago, I wrote about the dangers of running nuclear plants past their design life. Now, with accident cascading into accident, the operators are tiring and making mistakes, and all seems grim in Japan to constrain meltdowns in several plants.

My heart goes out to those attempting repairs. Things may turn for the worse as it gets tougher to spend time at the site, safely.

Update: Wednesday 8AM MDT

I'd written this blog entry on monday, actually, thinking it was tuesday in the US and wednesday in Japan. Shows how much sleep I've been getting.

Heroic civil engineering and disaster planning in Japan

2011-03-13T11:52:00.000-07:00

Shortly after I'd written a piece on my other blog about the amazing successes of civil engineering in Japan to withstand the 8.9 earthquake vs previous disasters, I got wind of another piece that was written by someone IN japan that goes into more and better detail overall, and a third, that talks clearly to the nuclear issues.

Universally - wikipedia and the bloggers have beat the conventional press hands down for accuracy on this nightmare. It's too bad that everything that hit print thus far is so off.

Beating Our bloat

2011-03-08T07:07:00.000-08:00

At my brother's wedding last week (way to go Steve!), one of his friends noted I had stopped blogging... instead, I've been making serious progress on this New Years resolution:

Collaborate more. Make a stronger effort to find people worth collaborating with. Use email more. Use usenet again. Push into the mainstream more patches - but logout at the end of the day - create some music.

Back in mid-November I'd first caught wind of bufferbloat from Jim Gettys' blog.

At first I was merely intrigued...

I had seen the kinds of TCP traces jg was getting while I was in Nicaragua (working on the wisp6 greenfield wireless mesh network), and in several cybercafes and hotels. I'd assumed then it was merely the tin cans and string connecting Nica to the rest of the world. I'd seen traces like this, in particular, a lot:

I didn't understand the effects on TCP of bufferbloat until I saw his traces and then his more detailed analysis with different tools.

This is a normal TCP trace:

This is a bufferbloated TCP trace:

It looks like an EKG on crack! (That's what Steve Lord called it, anyway). I envision this picture on a milk carton, with the caption:

"Have you seen this trace?

Login to bufferbloat.net to learn how to fix it..."

I got interested.

So, I re-ran his experiments, against the wisp6 router testbed. The results, under bad conditions (heavy rain), were horrifying.... 10s of seconds of delay in the routers (!@#@!#!)... and explained why NTP, DNS, ND, DHCP, and most other traffic had stopped working under those conditions.

Still, even at this point (late December), I thought it was a local, device specific, problem. I did a little patch to those routers and fixed it, (but good!) and went on my merry way, trying to cope with my other wisp6 problems of autoconfiguration, ipv4 in 6 encapsulation, ipsec, mtu size...

Then I saw the netanylzr data and watched and listened to jim's presentation about 3 times...

The diagonal lines are showing latencies - across paths that should be taking under 100ms to do anything - all over the world - measured in SECONDS.

I realized, finally, it wasn't just me and my devices and my little network in Nicaragua.

Bufferbloat was a global internet-wide problem, one probably growing worse, rapidly.

I got alarmed. If NTP, DNS, DHCP, ND, etc., start breaking we're in a world of hurt, but if TCP/IP starts breaking worse really bad things will happen...

I emailed Jim Gettys on January 10th about the mis-understandings thus far in the press that I'd been trying to correct, and volunteered to donate a pair of servers that I had lying around, and maybe write an article about traffic shaping... he told me I was exactly correct in my own analysis...

I'd met him a couple of times, we'd worked on the same stuff, like handhelds.org, X11, and OLPC...

...and so I found myself instead hacking ruby and redmine, getting multiple servers running, using my rock and roll promotion skills to get people all over the world in disparate disciplines involved, hacking kernels, fiddling with AQMs and new algorithms, reading 70+ theoretical papers, writing multiple pieces and wiki pages, making deals, swapping services, picking up dropped balls, making a ton of phone calls and exhausting my personal email address book to get bufferbloat.net to be a real, functioning entity, with developers, theorists and users from all over the world, and not a talk shop.

And the rest, is history in the making.

I still haven't got around to writing the piece about traffic shaping.

Basically, Bufferbloat (see FAQ) is a new name for an old problem (RFC 970) that has gradually been re-introduced over the last 10 years. It's especially bad in cable modems, 802.11n gear, FIOS, but also can be seen in just about anything that has a wide dynamic range (GigE switches hat do 100Mbit). It's bad, it's ugly, it's screwing up the Net, big time, and it's just a mistake that we've (as engineers and network designers) have all been making for a long time...

Head. Desk. Head. Desk. Head. Desk.

The Bufferbloat problem is almost as bad as Y2k... And more solvable. It's just that the Internet is so much bigger now than in 1999 that is intimidating. More cell phones are being added to the Internet every quarter than we had total users in 1999. There's also a persistent fear that it will get much worse, before it gets better.

So we've been lining up people to fix it ever since.

While doing all that, along the way, I came up with a good idea for a cosmic background bufferbloat detector that was extensively discussed on usenet, and the bufferbloat mailing list. Nobody found any holes in the concept which means (darn it) I'm going to have to code it up - or convince someone else to do so.

Good stuff keeps happening... there are nearly 200 members of the bloat mailing list now, John Linville just released a debloat-testing kernel containing not only a new algorithm (eBDP) for wireless, but two new AQMs and some driver patches. Doc Searls graciously loaned me his column for an editorial in Linux Journal's upcoming June issue... Vint Cerf loaned Jim Gettys his column for IEEE computer (due out in a few days), multiple other writers have chipped in... Theorists, coders, cats and dogs, all talking to one another on the mailing lists...

About the only flaw in all this activity of mine is that I've been so buried by it all as to stop blogging!! The effort required to write something for a more general audience is so much greater than carrying out conversations with the people I'm collaborating with presently on email and irc that I've stopped journaling entirely. I'm trying to fix that today, a little.

I've learnt that while journaling/blogging is important, even necessary, to the writer and his/her creative process, writing the history down behind the writing matters to no-one else. (I'm journaling today so that I can remember the timelines here)

Also, cutting the history from the finished work helps a lot. I just learned this trick from esr, who has also taken time out on irc to teach me more about writing in the last 2 months than I've learned in 10 years of blogging. (It also took 5 other polished writers - Evan Hunt, Bill Weinberg, Richard Pitt, & Jim Gettys, to tell me in no uncertain terms that I was doing some things wrong - for it to register. I've undergone a writerly "intervention". It was painful, but I'll survive)

I wish now, that I'd opened up my writing to a writers cabal 25 years ago, or earlier. I might have got a few books done by now.

Tomorrow (wednesday) I'm in open-to-all VOIP conference call about bufferbloat, with the freeswitch folk. Please join the call to hear more. Or check out bufferbloat.net.

After I gave up on SIP based VOIP (after working on it for 6 years), and gave my last presentation on it, in 2006, at Astricon, I'd had no idea then that a goodly portion of the problems I'd had with SIP were tied to bufferbloat. No idea what-so-ever.

Solutions seem feasible, across the Internet, for a whole new level of interactive applications after we get bufferbloat fixed. SIP phones now do IPv6, which solves a lot of problems, too. I'm seriously encouraged.

Sometimes it takes giving up on something, utterly, in order to make progress. It's been a zen 2011 that way. And also resolving to actually resolve your new years resolutions - works too.

All this said, I'm going to take a break from all this soon and write a bit about listening to, and making great music, and about an old, cherished concept of mine (and jeff stram's) called the jam-o-phone.

2010 - A Very Hard Year, thankfully over

2010-12-31T09:35:00.000-08:00

It was the best of times, it was the worst of times..

In 2010:

My wisp6 project got close to completion. I'm still writing it up. That's going to take months. I find myself fixing bugs, and working on the technical problems I'd encountered, like bufferbloat, far more than the write-up.

In April the Survivor TV show rented my house in Nicaragua out from under me - killing the album I was recording there with the Pobrecitos AND the wisp6 project. The property dispute related to that house cost me a lot of hair, weight, money, and friendships. It was a massive no-win scenario, there was no way I could make anyone on any side happy, including myself. I don't think I'll ever be able to write up what happened in those final weeks before survivor hit town; it's still too painful. Maybe I could fictionalize it and deal with my demons that way.

The result of that stress got me down to 180 pounds, finally, from about 190. It's about the right weight for me. I got hypoglycemic, but looked good, especially for a 45 year old. I was already in the best shape I'd been in in a decade. I swam for hours each day at that house. I miss the 2AM swims the most - far better than sleeping pills!

At the start of the year I'd ended a relationship that wasn't working. I feel very lost without a better half, but was even more lost with a worse half.

I left Nicaragua, looking and feeling about 80 years old. I don't even remember the month now. May?

I took a grand (train) tour of the US to calm down and get my bearings. I went from a crashing two month low to a really wild engagement party - I met up with a zillion old friends! and inside of a few weeks - although still very tweaked overall - was feeling much better - but it wasn't until August before I even felt halfway normal - and bad stuff kept happening. In particular, I got slammed with an 270k overdue tax bill (since cut to 24k after spending months finding and then filing the paperwork). The government still has the entire contents of my savings account, however. I shan't ever see it back.

I reconnected with a lot of old friends, helped a few out of trouble, and met in person net-friends and net-family I'd actually never met in person.

I fell into a brief love affair with my Android phone. It's broken now, and I don't care.

I looked for a new place to live and settled on trying out Colorado, if I decided to return to the US.

In August, some work I did 12 years ago turned out to be more important than I thought. I got in trouble for writing about it, too. I was asked to take the piece down. I didn't. A small rebellion, to be sure.

Prior to the tax bill I was planning to return to the wisp6 project, now I can't, except at a low, background level.

In September - during the rainy season, the worst time of year - I went back to Nicaragua.

I debated long and hard about sticking it out, about resuming my projects, about restarting my life there...

In the end, I packed up or gave away most of my stuff, said goodbye to everyone, had one last - perfect - surfing session. A lot of people I know there had already left for gr$$ner pastures, too. Survivor brought on a sea change.

I quit a gig that conflicted too much with my conscience, and lost another friendship due to that. I'd taken money and hardware for the job, and ended up giving all that way. My conscience remains conflicted.

I returned to the US with nothing but a suitcase, laptop, guitar, and a broken heart.

I erased the 60,000 words of the book I'm never going to finish in a fit of pique. I'm GLAD it's gone, I can think up new things with what I learned while failing to write it.

I didn't surf enough last year. I didn't play enough music, either. The band I'd been in broke up the year before, and I'd loaned out my bass on what ended up a permanent basis, and without that, my skills slipped.

I wrote two pieces of new software, gnugol and cryptolisting. Gnugol is shaping up nicely and a couple friends are helping out. I am loving working with other people with complementary skills, and getting complementary feedback.

I switched to a new blogging system, and went back to Emacs for as much work as possible. My productivity seems to be improving.

The week I arrived in Colorado I had a shot at good jobs at two big companies, but couldn't bring myself to sign the NDAs they had for the mere interviews. The legal language was genuinely frightening! There's too much left I want to write about and publish to cope with having a gag across my mouth and mind. I think I'm stuck at being a consultant for small firms; where I can negotiate a fair NDA - at least in the US, the IP regime has become impossible to deal with. The opportunities crossing my desk lately have been really trivial and un-interesting. Not for the first time, I'm going to have to make up my own gig, or find a way of promoting what I'm interested in doing out wide enough to find a match.

On the whole, this was probably the fourth most stressful year I've ever had. I went from the high of near success with wisp6 to a totally out of the blue and sideswiped by a reality (show) low. From happy and healthy and productive to vibrating all over the place.

I made many bad decisions this year. I've dwelt on them a lot more than I would have liked, too. If I could have the last 18 months or so back, I'd have done something totally different.

Right now I'm not adjusting to the cold of Colorado well, and lacking a car, can't go anywhere on a whim. It was easy - even pleasant - to walk 6km in Nica - not so much here. I'm back up to 200lbs...

I'm glad this past year is over. There were so many things I didn't finish - what I regret most was only getting halfway through that album, even more than wisp6. There's no way to finish - or even restart it, now.

I have no idea what will happen next year! It's a blank page!

At least, I survived survivor. I'm paying down debt and working towards having options. I'm enjoying coding up gnugol. I'm (badly) coping with the irony of having a great protools studio downstairs and not having except my (admittedly great) roomate to play with. It still hurts too much to listen to the half mixed record I will never finish.

I miss Nicaragua and my friends there a lot, and yet I think I'd like to try somewhere else, perhaps Brazil, or Spain, whenever I can find a way to do it. I feel like my time, healthy, on this planet, is getting short. I want to spend more time with friends and family, and less time with computers, and see more of the Real World, with my own eyes.

My New Years Resolutions thus far:

When faced with a difficult decision: sleep on it, write it out, get independent advice. If people pressure me to do what they want, and want an answer now, they're probably on the wrong side of the issue: Say no, more often, sooner, not “maybe”.

Collaborate more. Make a stronger effort to find people worth collaborating with. Use email more. Use usenet again. Push into the mainstream more patches - but logout at the end of the day - create some music.

Learn some more (Filk) songs on guitar. Work on the hard parts (drum tracks, mostly) and get them nailed.

In the late winter, take a trip, maybe go to a SF con, maybe go east, maybe go west, maybe go south.

Go skiing, once, to see if I still like it.

Finish something - anything - so that I'm totally happy with it.

Live as simply as possible. Love, learn, and be honest and true to myself.

Say less, and listen more.

Stay alert for new opportunities. Or make some.

Pay attention to the results of the Dawn mission.

Have a happy new year, everyone!

(even those that have been acting http://www.alternet.org/story/149369/8_smears_and_misconceptions_about_wikileaks_spread_by_the_media/?page=entirelike tools in the wikileaks saga)

So what else happened during the info-skirmish?

2010-12-17T15:43:00.000-08:00

Was a week long wonder, or something else?
The week of wikileaks saw “revealed wrongdoing, war crimes, corruption, hypocrisy, greed, espionage, double-dealing and the cynical exercise of power on a wondrous scale.”. And that was just the cables!

While the weapons of mass distraction were deployed: the US passed a hotly contested budget deal, which reinstated a less regressive estate tax. I wonder if anyone managed to actually read the bill in its final form?

Also, the Federal Reserve data dump took back seat to all the other news. What else did we miss, I wonder?

The climate talks concluded in Cancun, google released Culturnomics.

The launch of the space shuttle was pushed back 2 months.

The falcon 9 made orbit, and it's capsule, Dragon, landed, successfully, after two orbits. The coolest thing about that was Marty Anderson's *overnight* repair of a cracked nozzle skirt.

Some notes from the infowar

2010-12-09T07:03:00.000-08:00

I've been watching with horrified fascination the entire wikileaks thing, and taking notes all week, instead of getting anything done. Rather than just keep accumulating notes I'm just going to post this article as is and if anyone has constructive suggestions or good links to add, please comment.

Weapons of mass distraction, unleashed

As I write, there's over 13000 articles on wikileaks on news.google.com. It's damned difficult to find any that are actually about the leaks!!

Here's some pointers to the actual journalists's series reporting on the actual CONTENTS of the wikileaks cables:

* http://www.guardian.co.uk/world/the-us-embassy-cables The guardian cables.
* http://www.nytimes.com/interactive/world/statessecrets.html NY Times "State Secrets"
* http://www.spiegel.de/international/topic/wikileaks/ Der Spiegel (in english)
* http://www.elpais.com/documentossecretos/ El Pais (a chance to exercise mí Español or google translate)

And some knowledgeable journals have set up sites to discuss the issues.

http://wikileaks.foreignpolicy.com/ Foreign Policy (I found their investigation into the Zimbabwe diamond racket fascinating.

And then there's the previous War Logs analysis by the NY times, which I hadn't read before now.

I am delighted that some of the above have RSS feeds, so that I can subscribe to the content, and not the noise. (Getting this blog entry into a web format rather than an RSS format is proving a problem! I've basically had to abandon the web to stay on top of things)

I really miss netnews. Issues were categorized into topics and anyone wanting to read or post something on that topic did. You could implement - on your own machine/account, a filtering mechanism that rated the authors of the posts, so you could filter them up or down for factual-ism or bias. It was called scoring - and that's difficult to do on the web. The current architecture of twitterers and followers and fans is not as good.

There's also been some very interesting stuff from the horses' mouth:

http://www.commondreams.org/view/2010/12/07-1 Assange's editorial in the Australian, or the interview in Time.

And there has been some truly compelling commentary -

Glenn Greenwald in particular, here, and especially here, but also the Harvard Business review, and mother jones and the New American. I freely admit that this paragraph of links conforms to my own biases, however:

I try really hard to read original sources, and not the extracts and analysis repeated by talking heads, I only read newspapers and news-magazines and bloggers that have an open commenting policy (often the comments are better than the news!), and also turn to wikipedia's talk pages to try and gain real insight into the debate inside of presenting an issue.

I also try very hard to compare the news with my direct experience. For example I saw clear signs of a DDoS in Nicaragua at the time of "suicide" of Managua's mayor, and learned yesterday (via the wikileaks cables) that my private opinion of the Hondouras Coup may have been correct.

Lastly, I try to avoid groupthink myself by going to sites that I know hold opposing philosophies, such as redstate. I always come back from that pretty shaken, with a need both to recheck my facts, and take a shower, and sometimes, comment myself - but usually that's in such a poisonous atmosphere that sustained participation proves impossible. I'm still nerving myself up to make a comment about how good current cryptography is, over there.

Even with all those critical methods in place, I find it really hard to separate fact from opinion! I owe a big thanks to my reliable commenters to straighten me out when I go off half cocked.

These methods don't help those that still read paper, or watch TV news, they are all victimized by what I call the meme war: the quest, by all media, for a simple, convincing phrase that jams a complex issue into a pre-biased word-bucket.

The meme war has been going on, and escalating, ever since the invention of mass media.

99% of modern “journalism”, via mass media, seems to involves capitalizing on a key phrase and then endless repetition thereof.

Conventional media - particularly broadcast media - only has a fixed amount of words to fit any issue into - and no feedback mechanism exists besides the long, slow process of a letter to the editor, and the occasional printed retraction.

So we see, over time, various sides of an issue testing new phrases out on the public, and seeing what sticks, then repeating it endlessly as a substitute for refer-ing to original sources or a quest for the truth. So far I've seen "whistleblowing web site" mostly replaced and “hactivist” now competing with “terrorist”, and numerous other examples of a loaded word in nearly every article I read about the controversies.

As soon as you tag a noun with a loaded phrase, pre-thinking happens. The weapons of mass distraction still work, really, really well, and I hate that.

Simultaneously with all this nonsense going on is a giant budget deal with congress, huge revelations about the Federal Reserve, and no doubt other news that has been buried by the WMDs in play this week.

Keeping score on the infowar

Netcraft is tracking uptimes for afflicted sites.
Leakylinks reports, as I write this, that there are over 1411 sites successfully mirroring wikileaks, and over 14 DNS mirrors.
Paypal released wikileaks funds.
EasyDNS - after being mistakenly targeted by a DDos attack, decided to supply DNS to wikileaks, when asked. They say their customers love it.
2600 has come out with a well defined position.

On the Mass replication of wikileaks

Massive replication of network links and data was part of the Internet's original design - it's the only way to be nuclear war-proof - and as a side effect of that we got freedom of speech. And spam. And DDoses. It's not a perfect world!!, but I note that it is possible to build up your organizations infrastructure - whether you are a small or large business, to where you can serve your customers better in the general case. If you host your own DNS - have many copies - distribute your data regionally - and internationally - you can serve up more data, faster, with less latency, and less catastrophic failure modes - whether they be power failures, incompetency, someone with a backhoe in the wrong place, or a DDos,

Replicating services regionally is something that those afflicted by Anonymous - OR those that fear government sanction - or simply want to build a better service, would do well to emulate.

Other news organizations should explore the benefits of torrents and tor to see if they reach new markets, for example, beyond the great firewall.

One of the grave difficulties in doing data replication is that web content today is difficult with the complexity of the presentation stack - you need a database, a web server, some sort of dynamic content language, javascript, and and a horde of other specific tools.

Serving static content requires 1/1000th the resources a modern web site requires, for 90% of the functionality. It's why wikileaks is no longer a mediawiki. We'd be burning a lot less electrons and buying a lot less servers with more static content.

Cloud computing

I've been really critical of the cloud in the weeks prior to this, long before I ever heard of wikileaks. Now it sounds like a few others are "getting it", talking about the privacy of medical records, for example. There was a good piece that I can't find right now about localizing records like this.

I was unaware, until now, that the HITECH Act gives doctors $44,000 over five years to establish electronic health records.

I'm a strong advocate of privacy and security for everyone - and technologies like pgp and otr and distributed databases are some of the means to get that.

The DDoses

One overwhelming early narrative is that "wikileaks was under attack by hackers". That narrative was started by wikileaks themselves and picked up, without question, by every piece of news media out there.

What hackers? Whose hackers?

For over a decade now, I've figured any government worth its salt has been participating in a covert cyber-war, with multiple levels of plausible deniability. I can also imagine useful idiots, criminals, and cannon fodder on all sides with their bot-nets at the ready, ready to participate in whatever crazy idea a charismatic or well funded person or organization or government might come up with.

No hacker organization behind the attacks on wikileaks has come forward with the exception of a poser - So who attacked wikileaks? Is anyone investigating that?

Update: PLEASE NOTE: I am mortally opposed to the use of DDoS attacks for any purpose, by anyone. If technologically feasible I would deny this avenue of attack or protest to everyone -

Unfortunately, botnets remain a fact of life, and will remain so so long as insecure systems like Windows continue to exist, and third party apps like flash contain holes you can drive a truck through, and clouds remain cheap. I've been encouraging users for 20+ years now to be aware of security concerns, to not click on shiny objects, install and keep updated virus checkers, run adblock+ and noscript, and install Linux over their old Windows partition, or to switch to Macs... without much success.

On maintaining secrecy

Gary Warner (a very smart security guy) has one of the more cogent descriptions of information security lessons learned so far:

We can agree to disagree on whether Manning is a Patriot, an Anarchist, or a Traitor, but the important outcome of any event of this nature is that we document our Lessons Learned.

* Consider your own Information Collection in your workplace.
* What are the "Categories of Information" and how is access to those categories assigned?
* Within each area what are the "Sensitivity Levels" or "Classification" of that data?
* What is a "reasonable volume" for accessing data in each of those categories and classes?
* Perhaps most importantly, who is in charge of monitoring access to those categories of information, and how are "alarms" set when a category, class, or volume condition is reached?

Lastly...

Here's a plug for the often overworked, overtired, and certainly mis-understood security professionals and software engineers that have been working their asses off for decades to protect the computers of their friends, families, companies, organizations and governments from script kiddies, spam, and internet AND political attacks. 99.99999% of them are white hats, neutral - acting to keep the internet open for all. They defend, rather than attack. They work in an extremely complex, dynamic field.

It's a really high-stress job - it involves high tech skills, and eternal vigilance, with no upside - you get no respect or recognition if you do your job well, and take the rap if you get beat. Getting beat involves high stakes, too.

I used to do that for a living, until it burned me out. Defending against attacks and abuse like the "Ping Of death", etc, has cost me a lot of hair, and a lot of sleep. Instead of being on the front lines of defense, I merely observe them now. I have been working on new protocols (ipv6, sctp, hip) that I hope won't have the problems that the current Internet has. I'm not too hopeful about that, but to not try, would be to fail.

I'm very glad there are still people working hard at defending all our systems, technical, and legal. Otherwise, we wouldn't be having this conversation right now.

I'm also glad that, despite all the other weapons of mass distraction deployed this week, that spacex had a perfect launch and recovery yesterday.

I'm going back to work. (While researching new ways to defend and improve the systems of myself and clients, I discovered that - maybe - maybe- a key argument in the net neutrality problem was technical rather than political. It's called bufferbloat. I'm fiddling with the idea now)

Lastly (Really!), Laughter

Some outrages are best met with a giant belly laugh and outright ridicule. Here's a hysterical (and TOTALLY NSFW - oh, the irony!) defense of the first amendment. In response to homeland security doing a deal for an anti-terrorist campaign inside of wallmart, here's this response.

Seriously... going back to work now.

well, this presentation on how good modern humans really have it cheered me up enormously.

Update 2 (friday morning): I just realized I was deploying weapons of mass distraction on myself. ah, well, sometimes you have to let the world take care of itself for a while. I'm going to TRY to not look at any wikileaks output until tonight.

Practiciing safe hex

2010-12-07T03:36:00.000-08:00

Note to future possessors of crippling government documents: it would be best to practice safe hex. Or to be a monk.

Update: to be fairer to all sides, this deserves some thought, as does this.

And to be snarky, the king of Sweden has had a really great time on the public dime.

The new social contract

2010-12-04T10:28:00.001-08:00

There's an overriding law in the land now, the "End user services agreement", the EULA. Being accused of violating one is enough to see your service cut off.

Non-disclosure agreements and employment contracts are truly frightening things. Mortgage agreements are stacks of paper 6 inches high. The size of the federal register, I try not to think about.

I try not to think about all the EULAs I've clicked on, without reading, in order to use a valuable service. Somewhere in the (must be hundreds, by now) of the end user agreements I've had to navigate, I've probably clicked away the rights to my first-born, and any unique idea I'll ever have, should it prove profitable, my right to my last name, and god knows what else.

I took time out to read the few agreements I'm knowingly subject to this morning, that violating in any way would impact my ability to function in America.

I don't read patents, in the normal case, because of the triple damages rule, and because I find reading them very depressing. Every time I'm exposed to one, I'm non-productive for months.

Comcast

I started my morning's reading with Comcast. I wasn't aware of this clause, until this morning:

“10.2 Comcast may update the use policies from time to time, and such updates shall be deemed effective seven (7) days after the update is posted online, with or without actual notice to Customer. Accordingly, Customer should check the above web addresses (or the applicable successor URLs) on a regular basis to ensure that its activities conform to the most current version of the use policies.”

Comcast terms of service are 17 pages long. They just released an update to the terms of service on 10/25/10, which added the following clarifications to their agreement:

It's nice of them that running your own web server is expressly allowed, and I wholeheartedly approve of the anti-spam proviso. It's worrisome that running any other kind of server is expressly left a little vague, even if you have rented a static IP address. It's clear that - at least at present - I can run my own DNS and chat servers. :whew:.

But, at a stroke, they've banned wifi coffee shops and throwing a 12 dollar cat5e cable over to my neighbor to share internet service. Routing ipv6 around town with my nifty new meshed nanostation M5 radios... looks impossible. I wonder how airstream wireless, in Australia, and the various mesh networks in Europe are managing to innovate around restrictions like these? Do they have similar EULAs to deal with?

For "normal" comcast cyberserfs, it appears that many (most?) people are violating the comcast terms of service as they stand today. Anybody that uses dynamic DNS, or a squeezebox server, even a fileserver, seems expressly forbidden now. Is it against the new terms of service to have ANY other kind of server in your house if you don't have a static IP?

There's a special set of rules for teleworkers. I'm not sure what they are, because they aren't online. You have to call to get them.

If you want merely a job interview with comcast, you gotta sign an insanely restrictive NDA. Without a signature, there's no interview, and there's NO negotiation over the terms. After seeing their just-for-a-job-interview-NDA I'm sure their employment contracts make slavery or working in the food service industry look like a more appealing option.

Google

At least in mid 2007, Google would let you get away with not signing a NDA for the job interview, as C Scott Ananian fully documented his discomfort and experience at signing that one. He evidently didn't get the job, but at least he can still talk about technology, including google, at a deep level.

Google maps' terms of service are not horrible.

THEN there's google's terms of service for using their search API, which I'll talk to at another time.

Craigslist

Craigslist terms of use, um, makes my brain dump core.

I'm aware that the vast majority of provisos in all these agreements are rarely enforced, and are there primarily to cover corporate arses, but I'd like there to be AT LEAST the following things addressed in America's corporate culture:

1) All basic legal agreements (interview, employment NDAs, employment contracts, EULAs) posted online as a condition of being allowed to function as a corporation... people need to know what they are getting into.

2) There be some attempt at thoroughly vetting these vs a vs the actual laws of the land so people like me, without full knowledge of the law, do not have to be attached to a lawyer at the hip everywhere we go...

3) and some attempt at commonality, of thorough vetting by independent organisations - that would reduce the reams of paperwork regarding using any new service or signing any corporate contract to something sane. I'd like STANDARDs for contracts, in other words. A consumer reports for contracts. Something like that.

Personally,

4) I'd like, by law, that ALL NDAs in particular, should EXPIRE in a reasonable time - like, 3-5 years - except where national security is concerned, which should be more like 10-20 years.

5) Contracts should not be able to be changed on a whim and forward updating changes accepted as part of the contract.

I run Linux, exclusively, these days, because the terms of service in the Microsoft EULA are unacceptible to me, as are most Microsoft based products. I understand how the various licensing schemes Linux uses work - there's only about a dozen - very vetted by various court decisions - that I can rely on, and trust.

The one piece of commercial Linux software I have - fully paid for - is the cepstral speech synthesizer. After reading their agreement today, it looks like I can't distribute an example of the cool way how I use speech synthesis with my email notification system, which is too bad, I've been meaning to do that...

I don't know if there was an age, ever, where legal agreements so cluttered one's mental landscape. All I have is a distant memory of loyalty oaths during the McCarthy era. Now, THAT, was a simpler time.

After having my eyes glaze over on the first 120 pages of agreements today I decided to not look at blogger's and just post this piece. I remember reading blogger's EULA 8 years ago, and it didn't have a clause in it that required I read it again, then.

All my NDAs, except one of dubious enforcability, have expired. I'm glad of that. The EULAs though, are beginning to bother me.

wikileaks investigations from overseas

2010-12-03T09:33:00.000-08:00

I am really saddened that I have to get most of my journalism into the latest crop of wikileaks from overseas.

Wikileaks reduced to an IP address overseas

2010-12-03T04:22:00.000-08:00

At first I thought wikileaks' loss of their .org domain either a convenient excuse or an example of technical incompetence. I mean, if a DDoS could take out the .org root there is something seriously wrong with the people running it.

It turned out that wikileaks wasn't running their own DNS servers, but relying on the free US based service, EveryDNS to propagate their IP addresses. IF wikileaks was using their own DNS servers, and merely using a primary .org as the root, they'd be able to not only lower the load on any given root server - but be able to tackle and analyse the sources of the DDoS attacks on their own, which I think would be a very interesting investigation in and of itself. I'd like very much to see the sources of the DDoS published.

John Gilmore famously wrote, in 1993, “The Net views censorship as damage, and routes around it.”

That was in an age of Netnews, where tens of thousands of copies of any given message existed on tens of thousands of servers distributed throughout the world, managed by a network of volunteers, and the contents, more or less - covered by common carrier law. The Net, then, as we knew it - was impossible to censor.

What we call the cloud today is very weak in comparison to netnews, and putting your data in the cloud is subject to arbitrary constraints.

In our new, golden age of the web based internet, information tends to reside in one place, on only a few IP addresses, where the loss of DNS or web services is crippling. The Streisand Effect is no longer as effective as it used to be, particularly with the infrastructure required to build a modern website in early 2000's style - php, database backends, etc, etc.

I remained puzzled as to why wikileaks isn't using (whats left of) netnews, and rss, and tor (.onion), using blogger and things like github, and opennic to get their information out, and also putting up mirrors on IPv6, and signing up volunteer mirror DNS and web sites left and right. I don't know how their site is designed but it should be shipping out static html via some non-interactive wiki compiler and a database that's duplicable and other easily mirrored techniques like that to lighten the server load and make DDoS attacks less feasible.

Update: Since writing this post I've discovered that there are now a LOT of sites mirroring wikileaks. Here's one. Another site documents many more mirrors as well as a tor (only findable if you install tor) and freenet site. Still..

The 80 cable-a-day format of wikileaks's current strategy would be perfect for highly entertaining netnews newsfeed, actually, but I don't know if what is left of the netnews network can still function, or if anybody but me still reads it. RSS would be highly distributable as well, with a bunch of mirror sites signed up. I'm glad twitter exists, but netnews would be better.

While I have deeply mixed feelings about the content of wikileak's current stash, and feel the timing of the release is being used for other purposes (or, worse, is all the hoo-rah a distraction to take away attention from the shenanigans at the federal reserve? or the 9.8% jobless rate?), I feel wikileaks has brought good information to light in the past, and more is promised for the future...

...and I wish more of the Net story was about the content of the cables, rather than wikileaks's battle to stay online, and about how censorship, of any kind, is a bad thing, for all citizens of the world.

I can forgive EveryDNS for bailing on providing services to wikileaks, as they were working for free - but not Amazon.

EveryDNS posted yesterday:

Wikileaks's services were terminated for violation of the provision which states that "Member shall not interfere with another Member's use and enjoyment of the Service or another entity's use and enjoyment of similar services." The interference at issues arises from the fact that wikileaks.org has become the target of multiple distributed denial of service (DDOS) attacks. These attacks have, and future attacks would, threaten the stability of the EveryDNS.net infrastructure, which enables access to almost 500,000 other websites.

BUT As for this level of doublespeak from my own government:

R. CROWLEY: The official position of the United States Government and the State Department has not changed. We value a vibrant, active, aggressive media. It is important to the development of civil society in this country and around the world. Our views have not changed, even if occasionally there are activities which we think are unhelpful and potentially harmful.

QUESTION: Do you know if the State Department regards WikiLeaks as a media organization?

MR. CROWLEY: No. We do not.

QUESTION: And why not?

MR. CROWLEY: WikiLeaks is not a media organization. That is our view.

I would love for the state department to define what, exactly, IS, a media organization... or when it is, or isn't.

In our increasingly Orwellian world, I still (barely) remember the history of and ultimate fate of paranoia.com.

Bonus link: Glenn Greenwald, in full flower.

“...any attempt by political officials to start blocking Americans' access to political content on the Internet ought to provoke serious uproar and unrest...”

Update II: I feel remiss in talking about the controversy rather than the issues raised by the leakage so far. Here's a list (via greenwald, above) of the kind of stuff wikileaks is bringing to light:

(1) the U.S. military formally adopted a policy of turning a blind eye to systematic, pervasive torture and other abuses by Iraqi forces;
(2) the State Department threatened Germany not to criminally investigate the CIA's kidnapping of one of its citizens who turned out to be completely innocent;
(3) the State Department under Bush and Obama applied continuous pressure on the Spanish Government to suppress investigations of the CIA's torture of its citizens and the 2003 killing of a Spanish photojournalist when the U.S. military fired on the Palestine Hotel in Baghdad (see The Philadelphia Inquirer's Will Bunch today about this: "The day Barack Obama Lied to me");
(4) the British Government privately promised to shield Bush officials from embarrassment as part of its Iraq War "investigation";
(5) there were at least 15,000 people killed in Iraq that were previously uncounted;
(6) "American leaders lied, knowingly, to the American public, to American troops, and to the world" about the Iraq war as it was prosecuted, a conclusion the Post's own former Baghdad Bureau Chief wrote was proven by the WikiLeaks documents;
(7) the U.S.'s own Ambassador concluded that the July, 2009 removal of the Honduran President was illegal -- a coup -- but the State Department did not want to conclude that and thus ignored it until it was too late to matter;
(8) U.S. and British officials colluded to allow the U.S. to keep cluster bombs on British soil even though Britain had signed the treaty banning such weapons, and,
(9) Hillary Clinton's State Department ordered diplomats to collect passwords, emails, and biometric data on U.N. and other foreign officials, almost certainly in violation of the Vienna Treaty of 1961.

If wikileaks isn't a media organisation, then what is?

Will no one stand up for wikileaks?

2010-12-01T12:21:00.000-08:00

I applaud wikileak's continued, independent, x-ray exposure of so many skeletons in so many closets. Nothing makes a day more satisfying than seeing Ms Palin AND Ms Clinton, BOTH frothing at the mouth.

Wikileaks is so fundamentally an American institution, of a free press, of a quality journalism, that seems to no longer exist, that I'm ashamed, as an American, that the site has to exist outside the USA in order to stay up and viable.

I look forward to wikileaks tackling the banksters soon, and I hope that more journalism organizations grow back their balls, but I'm not holding my breath. Julian Assange, in a very intelligent, reasoned interview, nails the real problem in one:

“... In the United States to a large degree, and in other Western countries, the basic elements of society have been so heavily fiscalized through contractual obligations that political change doesn't seem to result in economic change, which in other words means that political change doesn't result in change.”

It's good that someone keeps trying.

P/2010 A2

2010-11-28T18:17:00.000-08:00

A natural deep impact...

Working on a new blog - NeX-6

2010-10-31T08:19:00.001-07:00

I've started prototyping a new blog, NeX-6. I will probably get off of blogger entirely, eventually.

While my intent was, originally, to import all the old content into the new, I've decided to just repost the good stuff from the past, there, over time, until I figure out how to get all the old information out.

If you read me via RSS, please add nex-6.taht.net to your RSS feeds.

NeX-6 had a few unusual design goals. (I wrote about them at more length here). More briefly:

0) Re-establish my presence on the web, searchable, under my terms.
1) Be mirror-able - to work on my laptop, especially, while I draft stuff. (I desperately needed to improve the revise/edit/revise/post part of my writing cycle)
2) Be FAST, even on small web servers running on embedded devices.
3) Work over IPv6 and IPv4.
4) Be readable on handheld devices (android/iphone/kindle) and by the blind.
5) Have an integrated wiki.
6) Keep a clear record of changes in source code control.
7) Allow multiple writers and editors.

In order to do all that I had to throw out blogger - and a lot of conventions - there's no database or php back end, for example. It uses a lot of CSS, and will eventually use a lot of javascript.

NeX-6 runs on my own servers and home gateway, not in the cloud. NeX-6 is not built around LAMP. It's built around GWIW - Git, me, Ikiwiki, and the Web. (another use for this TLA is "Give me what I Want") NeX-6 is web server independent, and although perl is used for editing it, it's not integral to serving pages, and I actually do everything from within emacs, not my browser. (In glorious, glorious green!)

As I write the commenting system is in flux and editing is disabled. I'll open those up shortly. In the meantime, if you have any trouble accessing the new stuff, please let me know here.

I really hate giving up the googlejuice the-edge has accumulated, but I suspect my posts here will become even more infrequent.

This is my tagged log of the ongoing creation of the design.

Who invented the embedded Linux based wireless router?

2010-10-21T11:38:00.000-07:00

In December, 1998, Greg Retkowski and I published the Arlan Wireless Howto. The wireless router we started building in March of that year, and later documented so extensively, is now considered by CISCO's lawyers to be prior art to the Linux based wireless access point, in the court case Optimum Path vs Cisco/Belkin/SMC/Netgear.

I was deposed in August, 2010 to talk about it.

Optimum Path is now suing basically everybody making an embedded Linux based wireless router for infringing on patent #7035281, filed September 13, 2000. This patent covers many features of the WRT-54G series (first shipped in Dec, 2002), and related products, from multiple other manufacturers, features that were built into the Linux mainline code, long before the patent was filed. Optimum Path now also holds a second patent - filed in 2005 - granted in July 2010 - which covers not only the ground covered by the first patent but includes content filtering (!?). I'm told this latter patent is not currently the subject of litigation, but it bothers me as much as the first - we were doing content filtering in Linux in 1999, also... Everybody was doing it.

In looking back on the heady years of 1998-2002, I'm now nearly certain that back in July of 1998, we actually created the first recognizable "embedded Linux wireless router". PLEASE: Note the word choice, there - embedded, Linux, wireless, router. Eliminate any of those words and you end up with a different product, from a different person.

Our first wireless router ran code from the Linux Router project, had 16MB of ram, a pair of wired networking cards, a wireless card, a 486SLC processor, and booted from a floppy. Later iterations booted from a hard disk or flash. While getting all that to work was hard, all the heavy lifting had been done for us:

Linux did routing almost from its inception. The firewalling code and the ip masquerading code were in version 2.0, maybe earlier than that. The first "Linux wireless routers" came from the authors of the wavelan and arlan drivers. The first "embedded Linux router" came from the Linux Router Project - and the truly heavy lifting came from the math guys that came up with the signal processing software/hardware that ran on a small board that could send and receive the data signals over the air, reliably.

But... as best as I can tell, from looking back on the emails, mailing list postings, and archive.org, the first "embedded Linux Wireless Router" came from the minds of Greg Retkowski, myself, and Everett Basham.

Our wireless howto, my wireless diary, and the June 2003 retrospective I wrote about the project are now part of the court case.

I'm bemused. My diary - and blog - and our internal emails over that period - jokes, misspellings, foul language, personal conflicts and all - are now in the public record!!?? Over a patent suit?

It's weirdly pretty cool, though, that someone would think what we'd built 12 years ago, was still important and interesting, here in 2010. It's upsetting to see Anthony Spearman and Anthony Tompkins attempt to patent so much work that was created by others. Did they ever try to build what they described? Make it work?

We'd started our wireless router project back in March of 1998 , and only got around to documenting it after it worked. We'd built the thing out of spare parts, to solve a specific need, and we never once thought about patenting the idea. Not once! We did spend a lot of time trying to figure out commercial applications for wireless, as you'll see from some of those emails.

In 1998, we didn't solve all the problems that patent #7035281 covers, but we did build a working device, then, that was very close to what was described in it. Later versions of our stuff - for example switching to 802.11b, and Linux 2.2, which had bandwidth shaping built in, came even closer - but we never got around to publishing anything about those versions. Others got even closer...

CISCO's lawyers have a stack of paper a foot+ high - that hopefully covers the features that were missing from our published efforts in 1998 and 1999. So many other people were working out the problems everywhere else in Linux! We had day jobs!

After giving my deposition, I've thought deeply about what happened in wireless and Linux from 1998 forward, and done a bit of independent research. I figure, maybe, by publishing what I know so far, more of the history and prior art behind the "embedding Linux in a wireless router" idea will come to light, and head off the second patent at the pass.

Or maybe it won't. I don't honestly know what constitutes an original, patentable idea in my field! I'm patently aware of where all my ideas came from, and I'm certainly ignorant (though learning fast!) of the legal processes involved both in filing and contesting a patent.

The initial legal processes and paperwork for contesting patent #7035281 are nearly complete. In February, the case goes before a judge to decide whether to continue, and a jury trial, if needed, is planned for the spring.

My mental question remains. Did Greg, Everett and I really invent the embedded Linux based wireless router?

All we did, basically, was take code that already existed, compile a new driver, install a board, make a few cables, and prove such a box could stay running in a world where people trusted IOS. We're just the first people that bothered to plug in a wireless card into a junked PC, boot Linux off of a floppy, run wirelessly 13.1 miles and then publish how to make it work, in plain english, a howto a more general public, and even a patent lawyer, could understand.

Amusingly enough, our little howto hung off the far end of that wireless connection for years, dissipating electrons in the airwaves, for every one of the tens of thousands of hits we ultimately got. Everybody ate from our dogfood, in other words.

I've helped do way cooler stuff than this in my life, but I don't think anything I've ever done occurred at such an inflection point, almost a singularity, of the history of networking. In 1998, only a rare few had a wireless connection to the Net. Today - 2010 - wireless is everywhere. Wires are passe'. How did that happen? What will happen in the NEXT 12 years?

The intersection between the legal system and the coding process is fascinating - what was obvious then? What was not? Does something have to be turned from code into English before it's prior art? It's been difficult to figure that stuff out. 1998 was the last year of the prehistory of the modern Internet. Archive.org was barely archiving at the time, USENET was dying, email was overwhelmed by spam, Netscape had just released Mozilla, and many of the modern services of the Internet, such as Google, were just getting off the ground. Trying to figure out who did what, when, ate a lot of my time last month.

So I ask - Who followed our recipe to build and deploy one? Who made it do more stuff? Who went their own way? Who made the jump to 802.11b before we did? Is there anybody out there that remembers what happened inside the 802.11 and 802.11b standardization processes? Who built businesses on wireless Linux? Who, at linksys, ultimately, decided to adopt the broadcom MIPS chip - how did their WRT54G come to be? When did that chip first boot Linux? Who at Broadcom got that to work? What OSes drove the other wireless routers that came out at the same time? Were Arm or PPC chips ever a contender? When did DHCP get radius support? What drove - who funded? - the people behind Busybox, dnsmasq, ipchains/iptables, - the core features that are in every Linux based wireless router today?

"With enough eyeballs, all bugs are shallow"

Is marriage a survival trait in the North?

2010-10-19T05:56:00.000-07:00

I am thinking that marriage is a survival trait. If nothing else, back before the days of central heating (which really wasn't so long ago in America and Europe), having another body next to you at night kept you warm! (this also explains a predilection for large families)

Single people freeze to death more often than couples.

I've been sleeping under 3 sets of blankets, which I left in disarray last night so lots of heat escaped.

This morning I woke up and put on a hat, two extra layers of shirt, long pants and two pairs of socks - and if I could type with gloves on, I would. And it's only 60F in the house this morning. I'd be completely non-functional below 50F, I'm sure.

The heat is out, the fireplace is faux (is there a law against real fireplaces in Colorado?), and I'm still freezing... I've been eating at least 50% more calories than usual, not exercising, and have actually dropped a half pound in weight. The 35 degrees difference between here and Nicaragua seems to make a real difference in my personal heat engine....

Re-establishing a presence on the web

2010-10-18T14:30:00.000-07:00

My online life in Nicaragua was limited by many factors, notably the 5 times daily power flickers and the twice weekly 6+ hour long outages, huge DNS latencies, and a 128Kbit connection... and ultimately, the lack of having any internet within 6km of my house. I got out of the habit of relying on any machine but the ones in my household, adopting things that worked well offline - git for my source code control system for example, org-mode for my own personal database of writing, rsyncing via streamripper hours and hours of internet radio, using podcasts for my news, and ipv6 to allow me to actually be a useful node in the web (I ran my personal email and netnews servers that way)

I personally find the relentless "centralisation of everything web related on a cluster in the USA" trendline worrying. Everything important seems to exist today in the cloud, where I think (and still think) that a great deal of useful services belong on the edge of the local home or business (email/jabber/sip servers as three examples, local filesharing as another - it blows my mind when someone hands me a usb stick with the file I need on it - and really bugs me that I can't drag and drop files, wirelessly to my Android - at present, anyway)

Left to publish I have several long-delayed blog posts and a speech I gave in Australia on the centralization problem. I just found a printout of the speech I gave then - and can't find the file now - I felt the speech was a disaster, but looking at the next now... it's not bad, actually.

One of the most compelling talks about lowering our overall dependency on the web was Jesse Vincent's talk, 2 years ago, about the "sd" based bug tracker, which attempts to do for bug tracking what git did for source code development. It is accessible to just about everybody that has problems doing their job when disconnected from the web for longer than a few minutes.

HOWEVER

ALL that said, the power of the web for collaborative efforts, when it is possible to be online nearly all the time, IS compelling, and now that I'm back in the States (in a place that has had ONE power outage in 5 years) I guess it would be best to stop swimming upstream and go for something more normal people can use and edit.

(3G is seemingly everywhere - which increases the number of places where I can get online well beyond the mere coffee shops and corporate offices it was when I left America. )

I have a great deal of ipv6 related information that I wish to publish.

Most of the online references regarding ipv6 are from another era, structured as single-author howtos, often out of date or inaccurate, and focus only on the most modest scenarios of usage, like "how to get a home router" working. Data and instructions appropriate for "how to get a small business with multiple locations up on ipv6" or "how to build an ipv6 enabled ISP", doesn't exist.

I was up to something far more complex and hard to describe - a purely ipv6 enabled wireless isp - I called it wisp6 - and it's looking like getting the pleasures and pitfalls of that effort out to the rest of the world would be a boon for all.

Ipv6 is an increasingly hot topic. Comcast is making great strides towards a deployment in the near future, with experiments running on all the major modes of ipv6 deployment - 6RD, AFTR, and 6to4 tunneling. I've seen an explosion of ipv6 addresses in universities in South America, and China seems to be making progress too...

Something more dynamic, and multi-user, such as a wiki, would be more likely to stay up to date and grow beyond my personal boundaries of knowledge. I'd also like to have some blogging software that better matched my personal writing habits. Blogger bugs me.

Ideally I'd like to find some content management system that would be easy to manage on my Linux laptop - and run on my edge hardware - and be mirrored on the web - that I could do bi-directional syncing to when on and offline. I'd settle for a one-way system - hosted on the web, mirrorable on my laptop.

So, anyway, I'm fiddling with mediawiki (which is what wikipedia uses) and liking the features and default look and stuff like that, but not much caring for the resource requirements. I don't understand why a "live" connection to a sql database is needed, for example, when you should be able to just "publish" the entire thing directly from the source code repository.

Ikiwiki seems to have potential, and I'm fiddling with that, too. I REALLY want something that I can export cleanly from org-mode, but don't think worg in its current state is suitable.

It may be futile to continue swimming upstream on these fronts, and I might just bite the bullet and adopt mediawiki + drupal, rather than explore more options than these.

Suggestions?

Looking for linearity

2010-10-14T16:10:00.000-07:00

I'm often unable to write a paragraph without having to write dozens simultaneously on all the different topics on my mind.

I've only been moved into Colorado for 4 days, and I've got a lot on my mind. Today has been rather scattered. I'm simultaneously:

1) Trying to organize the coming week. I have meetings with a potential client, two potential employers, and a lawyer.
2) Trying to figure out if I want to order a (15 dollar/month) static IP address from comcast and run 6to4 over it or stick to my existing hurricane ipv6 tunnels and also coaxing 4over6 to work again on my nanostation m5s
3) Getting co.teklibre.org delegated to my new house
4) Coaxing my open-rd to do more stuff (like web/email/split DNS/samba) for my new roomate
5) Writing up several more of my "interesting failures" and the "internationalization" series.
6) Trying to figure out what wiki software to use for wisp6, or what, if anything, I can find that will interface with org-mode better.
7) unpacking my stuff and figuring out what else I need to get, when. Things like - a car.

Org-mode in emacs has been a life-saver, letting me create HUGE outlines, but it's still difficult to pull together everything in the end. This blog, wikis, emails, and the other forms of output media I deal with don't co-operate well with org-mode. I find that the last edits of everything I write get done in the final tool. I have no good way to import them back into org-mode, where someday I might easily expand and expound upon them.

While I can write a coherent sentence in a few seconds, a coherent paragraph requires 5x the effort, a 1500 word article 10x that, and a book - fuggetaboutit. I marvel at the people that can write linearly from an opening paragraph to a conclusion, and hardly pause in-between.

My loss of linearity has its benefits. I'm able to multi-task VERY effectively, sorting a half dozen tasks (a couple big, clustered, compiles, notes on multiple projects, some code, the web itself, a bug tracker) into my mental deadline scheduler, until my brain buffer overflows.

Often, I just type and type on various subjects until I'm tapped out, just to keep the output queue flowing. Eventually, something as irrelevant as this piece is to what I should be doing today, jells enough to bother to publish.

But sometimes I need to be linear. That last, great push to get something clean and done, where I finish something and then let it bake overnight for a final proof-read... I tell myself to wake up linear, to not check my email or messages, and just get that last thing out the door... it usually works. Otherwise...

The only way I know to cope, when my consciousness gets scattered, is reams of pipe tobacco, and lots of exercise, while dictating into a portable recorder. The classic "drive to the gym, work out for 40 minutes, drive back" just plain doesn't work. Long walks, surfing, swimming, and rigid calorie control based on the nearest store being an hour away, work best. I've owned exercise machines. They just sit there, glowing, and make me feel guilty.

I get a lot of "tough, deal with it" comments, from fat people and people that naturally seem to be able to stay thin.

Over the past couple years, my carpal tunnel has receded to where I can play piano again. I don't ever want to lose that ability again, so I go to extremes to keep the workload on my hands at a minimum. I dictate a lot, use a really good keyboard, on a desk of a perfect height, and a footpedal. I can get a good 16 hour day in, without pain, so long as I sit back and dictate stuff rather than rest my hands on the keyboard while thinking. Then I can blast through typing up whatever I dictated, or, if it's big and I have spare cash, I can outsource it to a dictation service like Transcribr.

Although utilizing dictation services are a common thing for doctors and lawyers and CEOs to use, it's uncommon for a programmer or program manager. Whenever I get into the corporate world again, I figure I'll be carpal again inside of a year unless I occasionally outsource my typing.

Last month, I purchased a footpedal. It's the first one I've had in years. After fiddling with this one and getting the footpedal python code to nearly work well I've decided that I need another pedal - one for the common modifier keys like shift and control, three pedals under each foot, or maybe just two each.

I discovered that I liked having the pedals mapped to backspace and enter, but have no use for the center pedal. Should I make it be control, or escape?

Years ago I co-wrote a keyboard emulator in C for Xwindows, and it's better than the python based footpedal code by a large margin. For example, it would do repeated keys right, and run forever. I find myself restarting the python based footpedal code often enough for it to be annoying.

Even while typing on the best keyboard I can find nowadays, I keep wondering: Why is it so hard to have a split spacebar on the keyboard (backspace and space)? Why is that not standard? Is it patented or something?

I look at people texting rapidly with one thumb and wonder what their worlds would be like if they could communicate as fast as I do with just a conventional keyboard, much less one with a pedal. I'm really impressed by the quality of the suggested words on my android phone, it matches up with the desired word quickly within a couple letters. It's a lot better than autocomplete is in openoffice and just about as good as my emacs abbrev implementation (which as one example turns "dont" into "don't" automagically)

Ah, well, back to trying to get linear on the next piece....

Internationalizing myself

2010-10-01T10:08:00.000-07:00

One of the many reasons I've just spent 3 years in Nicaragua was that I wanted to see how the computer technologies that mostly originated in the USA were making it out to the rest of the world. Furthermore, I wanted to see how other cultures thought and operated, to see what they had to offer a citizen of the USA in return.

I'm always interested in improving communication in our ever more connected world.

I lost a few mis-conceptions along the way, but that's not what I'm going to write about today...

If you're tired of the technical detail I've been posting lately, you can skip down to the part of this blog entry that describes the personal incident that drives me to work on internationalization issues. Hopefully you will understand, and laugh with me, rather than at me.

Moving on...

People can talk a lot faster than they can type or text. Even a trained, professional, transcriptionist, equipped with special hardware and software, can barely keep up with someone talking at a normal speed, and dictation software is rarely up to the job, either.

But writing things down is one of the best ways to ensure that what you've said is clear - spoken words are often mis-remembered or mis-interpreted, so when it comes down to trying to communicate across professions or cultures or languages- writing stuff down is best.

I've just spent 3 years in the jungle, in part, trying to figure out how to make that level of communication better.

Coping with internationalization problems

A quick digression: In Spanish (Español) the letter "i" is pronounced like "e" - Is that an artifact of the great vowel shift? Was there some sort of transcription error when they started writing things down? Or what? (amusingly, I am not getting the ñ right on the web page, it looks like, right now)

Anyway. There's an accent on nearly every word you type in Spanish. This plays merry hell with the speed and quality of user input.

Text input device problems

I have a Spanish-specific keyboard, or so it claims to be. It has an extra key dedicated to the ñ key, next to the enter key, making hitting enter extra hard... And it has no easy way of entering the other accented vowels.

Under Xwindows (Linux/Unix), at least, there is a standard keyboard layout that solves the accent problem thoroughly and efficiently for most Indo-European languages, the "USA International (AltGr dead keys)" Keyboard model. It's one of thousands of keyboard models you can choose when installing a new machine.

By remapping the right alt key to being the accent key, typing in Spanish is made really easy - typing áéíóúñ is a matter of holding down the right-alt key and the letter. Capitals, right-alt+shift+the letter. ¿ is right-alt-?, and so on. I can make the euro sign (€) easily, too, but not the Cordoba...

Less common accent types (for other languages) exist nearby the relevant letter (on a querty keyboard) - the ä in my name, for example - is right-alt-q.

Easy, right? It was years before I hit upon using this keyboard model. If it took ME years, how long would it take others? I would argue that most of the Spanish speakers I know - have no idea how to get a Spanish keyboard model they can use effectively, either.

There are two other, more common keyboard models on other operating systems. In one - You hit compose, the accent, and the letter you want to type.

Other systems use "dead keys" where you hit the accent (twice, if you want it to stand alone) and then the letter you want to accent. The "hit the accent key twice to get just the accent" keyboard model raises merry hell with people, like me, that program a lot and need to use the keys ~'" on a regular, standalone, basis.

Try as I might, I was unable to train myself to use either of these methods.

Worse...

What I see most people doing is grabbing for the mouse, then finding an option for "insert special character" in their word processor, finding the right character, clicking on it, and then continuing to the next word - an enormous productivity sump - and one that doesn't work in a browser or most tools outside of a word processing program.

All OSes do have the ability to enable alternate input methods - a little popup window that works with most applications to enter individual characters and/or remap the keyboard somewhat...

But most people don't know how to enable it. I was just in a cybercafe where someone HAD enabled it - for Cryllic - and I had the devils own time turning it off!

Others resort to having the spell checker fix it, or (the most common case) simply dropping the accents from daily use and depending on context to determine the differences between words. In a language where "Si" (if) differs from "Sí" (yes), this presents a gradually increasing contextual problem for all but the best bi-lingual speakers.

... At least three quarters of the Spanish speaking facebook and irc population I chat with just drop all the accents.

I sometimes resort to using google translate (I used to use babelfish) to do the job. I'll write a paragraph in English, translate it - fix it up for problems with gender (Amigo, o Amiga?) or tenses or formality (usted? Tú?) - and move on.

Coping with keyboard input cripples the pace of human communication outside of English.Using the "USA International (AltGr dead keys)" keyboard model, as I do now, makes typing in Español as fluid as it is to type in English. Does it exist on other OSes?

I'm not going to go into the horrors of typing Hebrew and Japanese today - or texting... Gah!

Coping with Textual representations

“With enough eyeballs, all bugs are shallow”

On this quote, note the use of “ and ”. While typographically correct, nobody, except publishers, actually uses proper quotes on the web, mostly (I think) because there is no easy way to enter those characters.

I can't tell you the number of times I've copied and pasted a bit of shell script from the web that used some other symbol that looked identical to ", but wasn't, and had to do a global search and replace for it so that the code would actually work.

Normal users have taken over the core of the languages we use to interact with computers - particularly notable with spaces and things like ';" in filenames. Free-form ascii usage in multicast DNS also bothers me a lot. "Joe's Machine", oy, vey!

These characters were reserved for programmers - in the 70s and 80s when Unix was developed and it's hard to cope with them being everywhere else, now.

It would be nice if there was a universal "programmers" character set that common tools like shell scripting languages, perl, C, C++ and Java could all use, instead of ASCII.

Apple's opening of the entire character set to end users in the 80s made them happy, but wrecked decades of development in computer languages that reserved most of the special characters for programmers.

I know... "Tough. Deal with it. The Users have spoken." Nowadays, the methods for dealing with universal character sets are so fragmented and incomplete and domain or language specific that I often wish for a time machine to go back and forstall at least the more user-friendly character sets until we'd settled fully on one universal character set representation, be it unicode, or UTF-8 - neither of which were realized in final form for over a decade afterwards.

The famous "little johnny tables (" problem, html's use of the % convention for unrepresentable characters like %20 for space, IDN's incredibly convoluted Punycode representation scheme are all artifacts of the intense difficulty we still have in mapping human communication to computer based communication. Life was so much easier when programmers had all the extra characters to themselves...

but... given that we now have the *ability* to enter symbols like ×, ÷, ≠ and ∞... why don't we reserve some characters to programmers again??

What froze development of computer languages into the ascii character set? Did APL scar everyone that badly? Why does the language of math, in particular, have to remain so disjoint with the languages we use to program computers?

Why NOT a programmer specific unicode/UTF-8 character set?

Just to amuse myself, maybe I'll try writing a ≠ function in clojure. I wonder how it will break?

Digressions: 1) And sublanguages have emerged, like the language of smileys. ♥? Where's the standardization committee for that? Or ♫?

2) Why don't we have automated tools that translate between english dialects? Translating from British English to American ought not to be hard: you could easily on the fly correct trivial re-spellings like theatre to theater, and common differences in phrasing like "Put something in the bin (trash can)" - and something that could automatically translate from male english to female english would be a real boon to the world!

3) Is it hard on other operating systems than Unix to have them use multiple languages at once on a per application basis? While working on translating Ardour into Spanish, I ran the rest of my system in English, and merely set the LANG variable to es before firing up ardour. Being already familiar with ardour in English, having the Spanish version in front of me improved my sight recognition vocabulary enormously.

It made the pain of adopting Spanish a little smaller.

Partial Solutions I'd like to see

* Standardization on fewer, better, keyboard types and input methods
* Autodetection of locale on installation or upon logging into a network
* Translation bots for all the major chat protocols
* Spell checkers that can be set to multiple languages, simultaneously
* The ability to switch languages on a per application basis

Claude

Back in 1995 I'd started an email correspondence with a lovely girl based in Nice? Lille?, France - Claude Derieppe - who was studying English. (I'm weak on remembering how to spell her last name now)

Every couple of days for over six months we corresponded - I'd write long emails in English, she would send off short replies, also in English, full of enthusiasm and exclamation !!! Points. It was quite delightful to be making progress - having a bit of romance, even - with such a woman, so far away - from deep within my dark cubicle, while I slaved away at a 80/hr per week job at a startup. She was my lifeline to the rest of the "real" universe.

One day I wrote her a long and serious letter asking her about how she felt about increasing the intensity of our relationship, about religion, about having children, and about... Maybe... Um... Someday... coming for a visit?

... And I got it translated into French using some then-new translation software I'd just bought. I checked a few paragraphs against phrases in my French dictionary, and it seemed ok, so I emailed it off, without attaching the English original.

I know, you're laughing now. I can laugh, now, too...

Several weeks of silence ensued. She finally sent me an email back asking what was I doing talking about "Eating babies?" and numerous other mistranslations like that... She was mortally offended and stopped talking to me entirely soon afterwards.

I didn't know how bad the software was, and also didn't realize (until after that fateful message) that many email systems at the time stripped out all the accents on all the words I'd sent.

And I realized then that the differences in how humans conceptualize and represent language is one way that wars start and relationships disintegrate. Hell, the differences in how men and women use English itself - supposedly a common language - are real and well documented.

The tools that we use to communicate via computers have come a long way since then, but still have a long way to go. I giggle insanely every time I see Star Trek's "universal translator" idea used in ways that simply could not work with any level of forseable technology.

I do continue to revel in cultural differences and wish more people would do the same. Take our attitudes towards "public property" vs "the queens land", as one example, or the "right of trespass and camping" so common in Norway, vs the plethora of land use waivers, rights away and eminent domain common elsewhere...

I later immortalized the experience with Claude in my wistful song, "Cybernation".

Everything is grist for my mental mill, and everytime I get stuck on solving some internationalization issue in software, I flash on an image of Claude....

And these days, 15 years later, I regularly correspond with people all over the globe in multiple languages, with the aid of google translate. I'm equipped, now, with a little more basic knowledge about how human languages work, but I'm sure I still make horrific mistakes.

I deeply admire citizens of Europe for their mastery of two or more languages, because it seems like I will never have more than a 3rd grader's vocabulary or grammar in Spanish, no matter how more study I put into the language.

Mastering just one language, Spanish, makes mastering Java look like child's play.

The news from my navel

2010-09-26T10:40:00.000-07:00

Over the past month, I've been posting my most interesting failures from my personal backlog/stash.

I always try to write up a postmortem of my projects, whether they succeed, partially succeed, or fail, so one day I can learn from them. Recently I nearly lost this personal stash of depressing documentation (the hard disk got rained on) and I decided that it would be best to whip some into shape for the blog and get them out there, and try to learn from them, personally. Maybe others can too. It still hurts to talk about some of this stuff... but as Spider Robinson says: "Pain shared is decreased, joy shared, increased"

I'm also... in the middle of temporarily, most likely, permanently, halting two R&D projects that I had intended to spend 5 years working on, 3 years in. I'm a month into writing up the postmortems. There were plenty of successes, and more than a few failures. Given all the work I did, it will take me months more to finish writing up the descriptions of the projects and what went right and wrong... and I guess that it's easier to look at the other things I did in the past that didn't work out and finish writing THOSE up than it is to turn into readable text the reams of lab notes and documentation I currently have on what I've been doing for 3+ long, lonely years.

I tried to go into my last two projects with my eyes open, fully aware I was doing R&D and R&D, almost by definition, doesn't go the way you want it to.

But after a while, I got pretty emotionally invested in them and couldn't see the forest for the trees. It took an enormous kick in the ass for me to get out of the jungle (the survivor tv show rented the house I was living in out from under me) and gain enough distance from those projects to be able to see that I cannot continue at the present state of technology. I just spent 4 months traveling the US, trying to gain clarity, and find somewhere other than California, or Nicaragua, where I might live, and do something else, simpler, that I might succeed at, or leverage what I just did, notably with ipv6, in building out the rest of the Internet, elsewhere.

While in the States, I did some interesting consulting on a lawsuit concerning patent 7035281, filed in september 2000, which basically patents most of the features in a Linux based wireless router...

...for which my friend greg and I had prior art in 1998. Portions of my blog are now in the court record, including this piece, where I counseled Dave Cinege:

I'm no stranger to getting so wrapped up in a project that I confuse it with growing a child. I've done it multiple times before, and I'll probably do it multiple times again, until I actually get around to growing a child. It hurts to give up a project that isn't rewarding - but it's not a child - you CAN and SHOULD abandon it if it isn't working out, and fill up that empty space with something else.

As I talked about this - now 12 year old project - with Cisco's lawyers, and others, and reflected on all the changes in the world since then, and where we succeeded and failed, then, and what happened since, I kept thinking it was long past time to take my own advice.

The rainy season in Nicaragua is driving me nuts! I have no internet at my new (rented) home, 6km out of town, and the road I live on is frequently impassible, so I have plenty of time to write, think, and plan... still, I find getting to town, and on facebook, on occasion, is a comfort, given what I'm writing about and the size of the plans I'm trying to make.

I'm planning on either moving deeper in the jungle or to Colorado, in a few weeks. Mostly, I'm thinking, Colorado. I have friends and family there, I really enjoyed my visit there, it's a lovely place, the people are great, and there is at least some high-tech there that might need my skills.

It's been oddly comforting reviewing the four failures I've written up so far, years - in one case, decades - after they happened. Themes have emerged - being too early, or underfunded, Paul Graham's truism that "the best people didn't work for me", and totally unanticipated and project killing problems with the toolchains and chips themselves - that were completely outside my control and range of expertise at the time.

Gaining wisdom comes hard.

Probably the best news from my navel was that doing the write-ups - spending the time to coherently write and publish, in English, about what we did - as we did with the wireless howto - was probably the smartest response to success AND failure anyone could have come up with.

In most cases, eventually, the technology progressed until something like the original vision came into existence.

I will continue writing up the failures series - and I hope that others have the courage to do the same - but I'm going to take a break from these writeups for a while and try to pull together my plans for the future. Maybe I'll take the time to write up a few successes, too.

I've missed living in Civilization. Perhaps it missed me.

Three pieces of hardware I'd like to see

2010-09-26T10:01:00.000-07:00

In casting about the past as I have been, at all the stuff that was too hard at the time, I keep remembering good ideas, not fully implemented, or with new applications...

I'd like to see a:

Lighting safe, battery backed, POE adaptor

I'd like to see a POE adaptor that includes a serious lighting arrestor. I just lost an access point to a lighting strike and I'm out 84 bucks - plus getting the darn thing off the pole is a real pain in the arse.

I'd also like one that had a small Li battery - good enough for 20 seconds of power at minumum - this would cut outage incidents in Nicaragua by 80%.

Either is feasible. Both? I found a circuit that would do some of the job...

The return of the butterfly keyboard

The patents on the Butteryfly keyboard are due to expire, soon. It was the best laptop keyboard I'd ever typed on. I'm not the only one that would like to see that keyboard make a comeback.

Portrait mode laptop

For years I thought I'd like to see a laptop that had a portrait mode display - for people that write code and text in front of a computer, rather than watch DVDs. I don't need one now - I can just velcro an ipad - or any of the innumerable upcoming competitors - to a wall in any orientation I like, or maybe use something with one of these fancy new roll up displays.

Sometimes I use xrandr -o 1 and turn my laptop on its side. It really is nice to see a full page of text on the screen, for what I do.

Interesting Failure #4 - Not speeding up Libsamplerate in 2008

2010-09-25T11:36:00.000-07:00

My initial motivation for fooling with libsamplerate was that I was working on mixing down a HUGE, 12 hour, 12 track, 96khz audio recording, to a portable machine that only supported 44.1khz output, so I was constantly using libsamplerate to mix down and convert dozens of songs to something I could actually hear, and spending 40 minutes per iteration fiddling with my thumbs waiting for that to happen.

I also had a hardware audio server project coming up that looked like it might use the then-new Arm VFP instructions for samplerate conversion and I wanted to learn how to vectorize code on that processor. Not having an Arm 926 yet to play with I started playing with the Intel/Arm SSE2 and SSE3 extensions and various compilers (gcc, icc) and their options - to see what I could do on that chip series to improve matters.

It also bugged me that the default samplerate converter for pulseaudio - now a defacto standard - is speex-float1, which has a S/N ratio that I haven't been able to find.

You CAN hear the difference between speex-float1 and libsamplerate's medium to high modes, but unless you are a passionate listener to high quality (24 bit) music and video at non-44.1khz sample rates, it's difficult. The first thing I do upon getting a new Linux box is switch it to libsamplerate medium from speex-float1... I listen to a lot of music. The added computational overhead puts an atom on the moon, but bigger processors do ok.

So there I was, with too much time on my hands between mixdowns... With full source code to my problem children...

When I oprofiled the code, bells went off in my head - something like 99% of the mix was being spent in a pair of fairly small routines that looked very optimizable at first glance. In particular, they weren't using SSE at all! Even though the hardware could process two doubles at the same time through the pipe, the C code didn't compile down to anything even close to that...

So I rewrote those core routines using the SSE intrinsics and got them to work on both x86 and x86_64 mode. The generated code got really dense, but the instruction length got larger.

The initial results were encouraging, especially on x86_64. By taking better advantage of all the registers available, AND pushing two doubles through the pipeline, and eliminating/parallelizing several float->double conversions, I got anywhere from a 14% to 50% speed enhancement on my hardware. The x86_64 scaled up to more channels far better than the x86 did, too, showing that one of the big problems in the code was register pressure.

I also cut the C code size by a considerable amount with some pre-processor magic. I was HAPPY, and announced my initial results on the mailing list... and started work on making something more generic and general purpose than my initial (two channel only) hacks. Doing a CUDA version even looked plausible and I was dying to try working in that...

In digging into the generated assembly I found that the gcc compilers would arbitrarily reschedule software prefetching from where I thought it should be to where it did little good.

So I went to work on making the code better, reading instruction tables, clock counts, and generally having a merry time researching the darkest details of the x86_64 implementations of both Intel and AMD. I remember that instruction decoding was fascinating - I learnt one hell of a lot about how the various levels of SSE actually worked, and about the schism in instruction set extensions that occurred between AMD and Intel after SSE3, and about how difficult the C intrinsics were to actually use in a language that didn't have a native 128bit type, or conception of vector arithmetic. In frustration, I found myself reminiscing about APL one day...

Ultimately I produced a pure x86_64 assembly version which did up to 8 channels of samplerate conversion, and met most of my requirements, among many other things the average instruction length was shorter to fit into the decoding windows better... but it was difficult to maintain, debug, and link to, especially as it worked best as inline assembly, not as a function call, and it didn't outperform my initial attempts by very much.

So, as my final shot at the problem, I rewrote the pure assembly into something that could be inlined using gcc's inline assembly extensions. It used up the entire x86_64 register set (this always makes an assembly language programmer feel warm and fuzzy - and most people would be shocked at how few registers most subroutines actually use), BUT: it wouldn't compile due to not having enough virtual registers defined in the gcc compiler.

I made one small patch to the compiler to fix that, and filed a bug on it. (Which the gcc developers decided to ignore)

Then I started testing with more processors than I'd had on hand at the beginning.

A lot of oprofiling later...

On the then-new Atom processor, in 64 bit mode, the optimized SSE code was actually SLOWER than the unoptimized code. I puzzled over this for a while but it wasn't until I noticed that on an old Opteron, the code also lost performance when doubles were used in the highest quality samplerate conversion setting that I realized the source of the problem.

The size of the lookup table basically wiped out the L2 and L3 cache on any but the largest Intel processors available a the time. The large stride did terrible things to hardware prefetching, too. My hand placed software prefetches worked well on only a few processors.

So I'd goofed (at least for older/cheaper hardware) by optimizing for floating point pipelining performance where memory bandwidth was more key to the performance of the algorithm. I easily forgive myself for this - it was impossible to "know" this without doing all the work I did delving into the algorithm.

While a CUDA version would still be interesting to try, most CUDA hardware available then (and most now!) doesn't have a lot of support for floating point math using doubles... And I think memory bandwidth would still be a problem.

And... building a version that had sufficient abstractions internally to take advantage of the dozens of different CPU types - much less CUDA! - the core loops would have to be optimized for proved to be hard too. It required far more object oriented coding in C than I cared to do (icc has a built in method making the support of different instruction set extensions easier, and I would like it if gcc had the same capability), and some tricky and hard to maintain cpu recognition routines.

Finally I found that when I compiled the original algorithm for the normal x86 register set, not SSE2, on x86_64, that it performed almost identically to my extensively hand optimized code.

This was discouraging in the utmost. I gave up.

The memory bandwidth limitation would be even worse on the Arm. Although using the VFP floating point unit made using libsamplerate feasible in the first place, I felt that it would perform badly on that chip at almost any sample rate conversion setting. So in the end I dropped that portion of the project, too.

As I write that hardware project is still x86 based, and we ended up using a DAC that was capable of sample rates all the way up to 192Khz, natively.

I bought a 3x faster Intel box, finished the massive recording project, and switched to recording most of my source material at 44.1khz, 32 bit. It would be interesting to play with the code and compilers again, once things like the VFX extension become more common, and Arms get more cache... Maybe the compilers have improved some...

But I'm also considerably more deaf than I was when I started caring about audio quality and find it harder and harder to care as overall cpu performance has doubled again since I started working on this. If anyone cares, I'll dig up the bzr tree.

My most interesting failures, #3: VOIP

2010-09-05T13:51:00.000-07:00

In 2001, while at Montavista, I started working on embedding asterisk on various platforms. It was totally accidental! I needed something to test the sip phones I was working on, and didn't have budget for a commercial SIP server, so I got asterisk running one day on a 386 box, and later PPC, mips and arm platforms. I was mightily impressed with what asterisk could do.

The SIP stack I was fiddling with was ultimately released as the "sophia" stack. I don't know to what extent the stack got used, but Asterisk on the other hand, went places....

Over the next few years I got very enthusiastic about asterisk and VOIP in general, as did a lot of people. I switched from mostly doing embedded work on tiny processors to high end servers. I maintained VOIP links to all the main VOIP providers at the time, blogged VON, spoke at a couple astercons, and helped get a wireless VOIP provider, wiline.com, off the ground.

Back in November, 2005, I started work on an embedded VOIP PBX intended for a private jet aircraft. Noise suppression and echo cancellation on an aircraft is a difficult problem - you have a huge amount of background noise you need to eliminate just in order to get the actual sound out of the microphone, and you still have to echo cancel the rest. The (struggling) design I inherited was built around a Intel arm chip which had a little support for DSP instructions, with some co-processors onboard that were a deep, dark mystery. There was no way Intel was going to release information on how to make those co-processors work to J. Random Company! And without being able to use those, it was just a fairly fast arm chip with no special features worth mentioning

There was ALSO no way the (commercial) code this company had intended to use would scale up from being a multi-line SIP phone to being a PBX, either, so I ported asterisk (AGAIN) in a matter of weeks and managed to meet most of the feature set required in the contract that way. Conference calling worked GREAT, for example.

But I and nobody else on the team could beat the echo cancellation problem on the analog handsets. We eventually threw a big FPGA on the board in the hope we'd eventually come up with a solution. The project was canceled in March of 2006. Ultimately I recreated the asterisk port, and finally got around to pushing out the last few core changes to asterisk required to make it work on a various architectures out to the mainline.

I don't know if the competing project for it ever got off the ground; it was saner as it used digital handsets throughout. (If you think FCC certification is hard, try FAA!) But... soon afterwards a functional asterisk made it into openwrt, and people started regularly embedding the VOIP PBX services into NASes, wireless access points and related devices.

Getting echo cancellation solved was something the indefatigable and brilliant David Rowe, whose PHD thesis was in speech compression, put his mind to at around the same time.

In addition to slowly perfecting oslec, he designed and produced an amazing board that got asterisk working on a low end, yet cool processor - the blackfin - which lacked virtual memory and a floating point co-processor. He announced the project's existence in late 2006. My first email message to David, was something like: "You're crazy, but you're my kind of crazy". We got to be buddies.

In 2008 I spent a few months in Australia, visiting with David and his family. At the time he was working on the IP08. We also kicked around ideas for what became the Mesh Potato.

The IP04 had been a success at both the open hardware development model, and in reducing the cost of a small business PBX.

It had flaws however - with the CPU lacking a MMU it had memory fragmentation problems and was inherently less reliable than a CPU that had a MMU. Asterisk itself, at the time, wasn't so reliable either. Wedge in some sort of gui like
FreePBX into the limited memory available, and it would die... reliably. Please note - it is plausible to build embedded products without a MMU but it takes a lot more software development time and testing - and the IP04 IS useful without a GUI and as a product... and multiple, simpler guis than FreePBX have appeared for it - it remains, 2 years later an amazing cost reduction that nobody has matched since, and the software continues to improve.

Those issues didn't stop one vendor from trying to scale that hardware design up from 4 to 8 channels. Hardware-wise, this was pretty simple, just add more analog boards to the bus and you were done... But they'd neglected to analyze the software stack...

Under the loads were testing, the oslec echo canceler got to 5 channels and the whole box basically crashed, using up 100% of CPU. There wasn't enough oomph left over to run asterisk itself after coping with the echo cancellation problem, multiple voip connections, etc. , the problems started showing up at 4 channels and became pathological at 5.

By hand coding the assembly for the blackfin on the IP04 and using the onboard scratchpad ram, I sped up the oslec routine by about 25% - but it took an epic fight with the toolchain at the time to make it work at all. The loop control register was ill-supported, and without it getting reliably dumped on the stack, I couldn't make the parallelism inherent in the blackfin architecture function to it's fullest extent. The core loop was only 10 (8!?) lines of assembly, as I recall, but this core routine couldn't work as written, with threads, without revamping the entire compiler!

I filed bugs, and started work on getting a new uclinux blackfin toolchain functioning (I also wanted ipv6 support!) but left Australia before I got anywhere. Other people were working on it, perhaps the work was completed, I should check.

Even if I'd managed to speed up the code by another few percent it wasn't going to be enough to get the IP08 to work under the full workload we were trying.

Update The IP08 DID ship, and under saner workloads than what we were trying, works just fine - so David Rowe tells me.

Perhaps there are faster blackfins now, with more onboard scratch ram.... At the time I daydreamed of just embedding oslec into an FPGA...

Along the way, by optimizing oslec's code path for modern architectures, I halved oslec's run-time via some pre-processor magic. While that is a worthwhile optimization for the thousands of asterisk servers out there, I still haven't bothered to get that into the main oslec tree or into the Linux kernel. It doesn't seem worth it. Analog telephony doesn't matter much anymore - the VOIP world is going digital anyway. One of these days I'll dig up the work. Or maybe David Rowe still has it, or someone else has done it already.

Update David tells me that many of my mods were not bit-accurate, but I do remember this last as being bit-accurate...

If ipv6 ever gets off the ground, getting sip to work the way it was designed to work would be worthwhile. But as things stand today, Skype works so darn well and is so ubiquitous that there seems to be no point in continuing working on sip based VOIP. Yet, there are many features of asterisk and freeswitch that I like - voicemail to email as one example - IVR - I used to have a converter for podcasts into on-hold music...

After using skype almost exclusively for the past 3 years, I am seriously annoyed at how bad the quality of the average cell phone connection is in the USA and in Nicaragua. While I was in the USA this past spring, I tried both AT&T and Verizon on multiple phones and was beset with difficulties - even going within their own networks.

"...an you bssshhhtt e ow?"

Quality is a metric nobody's figured out how to charge for. Maybe one day people will become so disgusted that they will en mass migrate to something else. Some, like me, for years now, have just leaned up against a nearby wireless access point with a Nokia n800 and made a skype call.

My verizon Droid won't transmit skype over anything but 3G. Skype over 3G sucks just as bad as the cell service does. I didn't know that skype had been crippled when I bought the phone, and if I had a choice I'd return it for that reason.

If I could, I would probably discontinue the voice component of the cellular service entirely. The data features (just the integrated GPS with biking directions!) are awesome enough as they stand.

TransConf

Sometime around 2005, I started working on what I thought was a genuinely new idea - conference call transcription - something that had a sound market need - how many meetings with 4, 8, 12 people or more, have you attended where you couldn't remember who agreed to do what? Or a good idea got arbitrarily rejected because it was out of the blue, so new, that nobody understood it at the time, and simply didn't register?

I called the enterprise, "TransConf".

I got stuck on multiple things: The normal (for me!) Lack of funding, and infrastructure, and an inability to find the words to describe a concept that only newly existed. While working on the idea, I found especially discouraging, watching people use freeconferencecalls.com for everything, eliminating the core revenue opportunity - I saw transcription as an add-on, and better conference calling general a highly desireable feature.

I also observed (disgustedly) that many people wanted plausible deniability for anything they'd said in a meeting and DIDN'T want anything written down, or searchable.

I got also got stuck on the software: I wanted a voip conferencing solution that would spacially locate the various individuals in your head, with stereo processing and a head transfer function.

I tried to produce (with ardour) something that handled multichannel audio playback well, with a pedal, to aid the transcriptionist, and ardour wasn't ready yet (and still has no decent windows version).

I'm using a pedal and ardour now for similar work, so ardour is ready, asterisk and freeswitch are ready, but is the world?

I let the domain name expire years ago.

Working on VOIP related technologies was a fun and cool phase in my life, and I'm glad to see others continue to work in it - not only on cooler, cleaner codecs, and more robust servers like freeswitch, but embedded devices like the mesh potato, better phones, video, and so on....

My most interesting failures, #2: The pre-ipad

2010-08-31T17:45:00.000-07:00

Somewhere in 2000 or 2001, while working for MontaVista, I got ahold of a webpad design built around Transmeta's intel clone chip. By 2001, I had it booting from (32MB!) of flash, and running Linux, Xwindows, Mozilla and an mp3 player in 64MB(!) of ram. It had a virtual keyboard using a hack from the Xtest library. It had wifi - I used it to stream internet radio all the time, in loving memory of the Kerbango Radio. It was easy to hold in your hand and on your lap - it had a much softer edge and was lighter than the ipad, as best as I remember.

It was cool. Oh there were flaws: The battery lasted about two hours. It also cost - retail - over 1500 dollars. It was slow. It didn't have bluetooth. The onscreen keyboard was a bitch to type on. It found a limited market in the medical field, but nobody I talked to could see the potential I saw in it, if only it could do more stuff, wirelessly, and had more memory, both flash and ram. Back then 32MB of flash cost serious money.

I showed it to everyone I could find - My bosses - Sony - Nokia - NEC - Apple - Mozilla - and it went nowhere. All people could see was a slow, and very expensive laptop without a keyboard, even though I would plug in (and velcro) a logitec wireless keyboard on the back. It sat by my bedside, velcroed to the wall, for a year, before I had to give it back.

Later on we starting seeing stuff like the smart door for meeting rooms, and dynamic picture frames, but they were very specialized applications that used less general purpose hardware.

Recently I waited - with great anticipation, for the crunchpad. I was upset when I heard that they were using intel architecture - the low battery life and size of the chipset were going to hurt them - I could have told them that by using Intel architecture they were barking up the wrong tree, but nobody asked me, and in the end the project disintegrated before shipping due to internal politics. With the right confluence of circumstances they could have beat Apple in many ways.

I finally got a chance to play with the ipad last month. (I know I'm behind the curve on this, but the nearest Apple store is 1200 miles away)

First impression: It needs velcro, and it's heavy. Maybe I could hang one from a rope in the ceiling so I can watch movies comfortably, lying flat on my back. Second impression:

Slam it hard up against a wall at a good focal distance and I'd use the hell out of it; get some desk space back. Maybe it works with a bluetooth keyboard. I have one of those around here somewhere. Or maybe someone makes a dinky little usb slave to bluetooth adaptor so I could plug in ANY good keyboard, like the , and make that talk to it. I've googled for that little box - no luck. Sounds like a market segment someone (else) could address... I tried my bluetooth headphones on it, they didn't work worth a damn.

And while so many devices support bluetooth, the technology that logitech uses for their wireless keyboards and mice is much lower power and longer range, and more reliable. Why haven't people licensed that?

But the ipad is cool. I'd get one if I wasn't broke, and the screen was a little larger and it came with velcro. I expect that we'll see a lot of competitors in the next year.

My most interesting failures, #1: Anagram architecture

2010-08-31T17:32:00.000-07:00

Back in 1993 I came up with a paper design for a multicore CPU architecture that was strikingly similar to SUN's Niagra architecture, which came out over a decade later. My design probably shared some aspects with multicores that shipped before Niagra did, too.

I wrote about it publicly, only once, but by the time I stopped working on it, a year later, in Dec, 1993, it was well developed enough to file a few patents, had I chosen to do so, but I didn't. Someone that "knew" that the crossbar switches inherent in the design were impossible to build, convinced me I was barking up the wrong tree. (That person worked for SUN, amusingly enough)

It used an ISA roughly derived from the MIPS architecture and a stack based register set much like the transputer T800. Probably the most innovative feature from my perspective was that I used RAMBUS chips hooked up serially, rather than in parallel, and divided the memory up into as many 8 bit paths as I had pins available. Getting the RAM industry to produce sticks of ram in this segmented fashion would have been nearly impossible! Soldiering the chips directly on the buses was my answer but this put a hard outer limit on the amount of memory you could have with RAMBUS....

Still, I figured that the narrow memory buses would be a way of producing a series of CPU chips of various performance levels. I thought, also, that you could eventually produce smarter ram sticks - with cache and a MMU on board each one - if you did things this way.

The design also could do DMA at the virtual memory level, rather than at the physical addressing level. Memcpy was basically an op-code that suspended that CPU and handed off the operation to the MMU. This took best advantage of the slow memory buses because it could do burst transfers - so common in the message passing operating systems in academia at the time.

I called it the Anagram Architecture because, being stack based, and highly threaded, you could rewind and restart various sorts of instruction in a super-scalar fashion, and run more jumbled up processes in parallel, more simply than you could with a classic Van Neuman architecture.

I don't know when the crossbar switches I envisioned became buildable, but they are everywhere now. I've dabbled with hardware design ever since, and played with a few FPGAs, but ideas like 32 bit and 64 bit memory busses are firmly embedded in everything, and developing a competitive CPU from scratch a pointless, expensive, exercise. Not that that stops people. Two interesting CPUs I've seen recently are the Propeller and David May, creator of the Transputer, is working on Xmos.

singing poorly at 45

2010-08-12T07:46:00.000-07:00

So, for starters, have a listen to my freshly recorded, for 2010 Rhysling and me. Try not to get depressed. At this stage (17 years after I wrote it), I'm mostly trying to create ART rather than cope with depression. I was amazed at how the Apollo 15 samples I found could be used to tell the story better.

I figure I will actually have the song DONE, completely, and PERFECT, by 2012.

I spent my birthday baking in the Denver airport, and on a plane, and then on a train platform in SF, shivering.

On this day I try to write major retrospectives on the past year of my life as I try to figure out what to do in the next year, that are going to be different, and better... but that involves looking at my last year of blogging - and the Internet hasn't reached 30,000 feet yet for Midwest. My external memory was offline.

The Net seems to have reached everywhere else in the USA, if you can afford 3G phone service. There's still 3/4 of the world's population left to hook up, however, most of whom need food, medicine, and shelter more than phone service.

I do hope that a few people, look up tonight, for the Perseids, meteor shower. I'm in a place tonight, where I can't see them, a place that I never thought I'd be in, but that's how my life has been going lately. More later, maybe.

Sax in Space

2010-07-20T08:43:00.000-07:00

In 1992 I met Kurt Heisig, in Santa Cruz, California. Meeting him, and seeing his story of how he worked with Ron McNair to create a sax that could play in the rarified and zero-g atmosphere inside the space shuttle, inspired me to finally compose the song that captured all the joy and pain I'd felt at every stage of man's quest for the stars.

I spent some time this weekend in Craig Guessford's studio, attempting to lay down a new version of that song (shorter, more intense), but didn't finish it in time to post today, the 41st anniversary of the first Moon landing.

I play Rhysling and Me obsessively sometimes. I remain haunted. What I wrote about the Challenger disaster is perhaps not as haunting as what the late Jeanne Robinson went through, or what Jarre had composed... but we all cope as best we can.

I dug through my storage unit last month and found a copy of Kurt's original, mimeograph'd story, and got craig to take this picture of the picture, of the only remaining documentation that a sax (and sax player!) actually flew in space. No recordings exist from Ron McNair's first flight, and from the second flow only tears, and versions of the song he was to play, recorded on the ground.

What's happened in the newspace program in the last few months has been incredible... from Hayabusa's successful landing, to the perfect launch of the Falcon 9, to SpaceShipTwo nearing it's first drop test... all give me hope for the first time in decades that Man may move beyond earth orbit in my lifetime. Perhaps after that, I can finally put Rhysling to rest.

Craig was sad that Constellation had been canceled, before I told him about all the good stuff that has been happening lately, outside of NASA. Now he's jazzed again.

I keep thinking maybe, just maybe, as we get restarted into the Solar System, I can do something a little more constructive for getting humanity off of Earth, than writing music. The highest an effort of mine ever got was a 160 miles above the earth, where it now floats in the atmosphere, evenly mixed with the molecules of James Doohan. Maybe I can do better next time.

the long slow, train ride, continued

2010-07-13T10:08:00.000-07:00

While I've been facebooking away on my trip through the USA so far, shooting a lot of pics on that cool android phone, I've been kind of remiss on typing up anything for the blog. I've mostly been accumulating experiences - for example on the train ride from California to Vancouver I recorded two excellent musicians - one, retired, in his early 60s, who played some lovely finger-picking beatles stuff, and another that had the classic early-20s grunge sort of attitude.

I need to find the time to mix all that down, and I still haven't had time to even look at Carlos Canales gig, which was nearly 2 months ago now, in LA.

So, anyway, so far, I went from Orange County, Ca, to Santa Monica (spent time with Secent), to my old hometown of Santa Cruz (among many other things, did a BBQ and potential podcast with my old colleague Bill Weinberg, the Linux Pundit), to San Francisco, where I caught the Kronos Quartet.

Then I hopped on the train again, and visited my distant cousin Aaron in Seattle (the Experience Music and Science Fiction museums were great, and I got his website working again, with IPv6 thrown in the bargain), then Angel from my old band said: "come to Tofino, BC - I MISS YOU - it's close" (NOT!) - so I went to Canada via train, bus, ferry, and bus... and spent a few days resurrecting 2/3s of our old band, and THEN... got on the train again and headed to Colorado, where I caught up with my childhood friend that shot the picture above, Chris Randall, and climbed a few mountains.

I have always liked Boulder, it seems like it is connected via trans-dimensional tunnel to Santa Cruz and Berkeley, but never found the right door to get me here in the summer...

I am now visiting with my cousin Brian and his lovely family...

Theoretically next week I'm heading to New Mexico to visit with my aunt and uncle and also my old friend Stu Zimny who plays in a symphony there (oh, man! have I caught up on Kulture!),
and AFTER THAT...

I intend to STOP someplace for a while, and get some work done. Really.

Unwired through the US and Canada

2010-07-05T10:21:00.000-07:00

Just got back from visiting with Aaron in Seattle, and Angel in Tofino.

Now I've got my feet stretched out, a g-enormous coffee mug on the fold down table, power for the laptop, the android on tether, connected to the Internet, and 54 hours of travel ahead of me.

I have, on facebook, sorted out the pictures of Tofino, Canada, and Seattle.

Perhaps, since I'm repeating the first leg of this trip in reverse, I'll start getting some work done. I have ONE project I NEED to finish...

Can I post this at 60mph while looking out over the bay... ooops, we started going through a tunnel just as I said that... ok... AHA! online again....

Couch surfing and train tour of the US

2010-06-23T14:28:00.001-07:00

Well, after an adventure with the survivor TV show, too long to publish now... (briefly - they are filming this year's show near my hometown of San Juan Del Sur, and rented out everything on the beach - including the house I was in!)

This was a major interruption in my planning. I was trying to pull together a field test of some new (300Mbit! 5.8Ghz! 12km range!) 802.11n meshed wireless radios - to spread internet and education into Nicaragua before the rainy season. I was running the radios out of that house and had 11 more to install on multiple towers and sites down there (email me if you want a google earth map) and the rainy season started early, and I had to really scramble to get a new house and cope with some other major complications, but eventually...

I rented out a new house in Nicaragua, which wasn't fully ready... so I came back to the USA for the first time in 3 years to visit my friends greg & cherie for their engagement costume party. (pics here and here)

And then I bought a nice used T61 laptop, had the seller slam ubuntu 10.4 LTS on it, AND I bought an Android phone, which works tethered, giving me internet anywhere in the USA.

Wow.

I completely missed out on two generations of Moore's law. I can, suddenly (from my perspective) call up public transit directions on my phone, and other maps, and take pictures via facebook and upload them immediately, and twitter (I'm http://twitter.com/mtaht over there), and chat, and skype (davetaht), dictate into a recorder, and zillion other things, from anywhere.

After stepping out so abruptly from Nica, I've now realized I could realize a childhood dream of riding the roads, and still get some work done on getting my multiple projects launched... not just the field test in Nicaragua, but maybe, just maybe, get somewhere on getting a real asteroid exploration program going.

I spent most of my life locked in a cubicle! I have seen so little of America, and always wanted to see more, both from ground level and from space. I've got my whole life, including 3 of those radios in my backpack, so I can "war-train" and see what 5.8 ghz spectrum looks like here, too..

So... I've been traveling ever since that party, with stops (so far) in Santa Monica, Santa Cruz, and San Francisco.

The Amtrak train up from LAX to SJC was Amazing, as it cruised over the beaches near Santa Barbara.

Sunday, for Fathers day, I visited with Bill Weinberg, (who I worked with at MontaVista), the Linux pundit, and scored a great interview, advice, and BBQ, with his family.

Last night, at Yoshi's, I met up with Anthony Adams from techweb, and a few other cool people, had some wonderful wine, and then caught the Kronos Quartet. Android almost convinced me... but this concert convinced me, finally, that civilization was worthwhile.

Tonight I'm heading up to visit a relative separated by 4 generations and reconnected by facebook. Monday... I don't know. Maybe back to San Francisco, or Santa Cruz... or LA, or Arizona, or Texas... I have 60GB of audio files to sort through, 2 new records to mix (here's some VERY EARLY sample tracks from one), 5 interviews to clean up and cut down to size, and tons of writing and coding stacked up, and revenue from all that just waiting for me to have the time to do it.

I can do it ALL on a train, now, with stops along the way. It would take months for me just to finish the backlog! I don't really feel like settling down in the US again, yet, AND also want to avoid the monsoon season in Nicaragua, which lasts through November....

I just bought a 15 day travel pass. Almost bought a 30 dayer.

And if you are anywhere near a train station in the USA and want to say hi, do a meal, talk about IPv6 and/or finishing the internet to the remainder *4/5ths!) of the world, or play some music, or talk about the private space program and/or asteroid exploration, and/or have a spare couch... let me know. Maybe I'll be passing by, soon.

I'm also looking for grad students/surfer dudes and dude-ettes that might be interested in helping me finish getting these radios deployed, using my new house in Nica as a base. And: Linux geeks who would want to work on the cool weather station chipper's been working on in combination with the Guruplug... Free rent... cheap food... GREAT waves http://www.nicaraguasurfreport.com/reportlist.php?id_secc=3&x_date=2010-06-20&z_date=%3D%2C%27%2C%27

My new 12-string (I named her Halcóna) and I have a bunch of new songs to practice, too. In my bag already are "Me and Bobby McGee", "Comfortably Numb", "Wish you were here", Dock of the Bay, "Last dance with Mary Jane", and a half dozen originals. Mauro Caminiti wants me playing bass in Nica with him when I get back, and his setlist is huge, with classics like She caught the katy, and "The joker" left for me to learn.

See you on the Rails!

Video for a manned Near Earth Asteroid exploration mission

2010-06-18T13:09:00.000-07:00

By Bruce Damer's digitalspace, who just made boingboing today.

Asteroid Delivery Service

2010-06-14T22:42:00.000-07:00

The spectacular recovery of the Hayabusa, and the perfect flight of the falcon 9, I mean, um, wow.

The video server was so jammed up on the falcon 9 launch, that we had 3 separate laptops all trying to get the live video, and ultimately only one with a very bad audio feed in the last seconds before launch.

I hit my recorder to get the last 20 minutes of my reactions on SD card. (It's around here somewhere) The frequent drop-outs added to the tension. I remember rooting for stage separation like some root for the Phillies, and when it happened it was like winning the season, and heading for the playoffs.

a launch too beautiful for words

2010-06-04T18:58:00.000-07:00

Carlos Canales plays LA, SF, Vegas in May

2010-05-16T21:48:00.000-07:00

I am freshly out of nicaragua and visiting the states for a while.

My friend and one of the best guitarists I've ever had the pleasure of listening to, Carlos "24" Canales, of Nicaragua, where I've lived for nearly 4 years - is touring the USA briefly May 28-30, as part of a pretty famous band (in Nicaragua, at least) in SF, LA, and Vegas. They're called Las Nuevos Panzers - and, man! are they great!

If you are in any of these towns and want to hear (and dance to) some totally unique music, well, here's the posters. Please pass 'em around. I hope to make the SF show myself.

I think there's also youtube video available too.

Check it out!

Fighting poverty via SPAM

2010-04-19T05:57:00.000-07:00

I don't get a lot of spam these days, but a few slip through. This one from this morning cheered me obscurely, as it was different from the usual stuff.

It set me to daydreaming of a world-wide poverty fighting team, equipped with ATM cards with giant balances, in the field, doing good, where-ever they could, air-dropping into troubled areas...

A next generation, post-modern, post-internet set of superheroes, working undercover, anonymously, to make the world a better place.

It would be nice, if it were true... cost-effective, too, if whatever was identifying the ATM recipients was actually researching the backgrounds they had, and otherwise keeping the staff at HQ low to non-existent. Maybe they'd built up an AI, that did all the research work.

Perhaps I'll write some fiction along these lines, one day...

FROM: dr.felixwilson@hotmail.com

Attn: Beneficiary,

I am directed to inform you that your payment verification and confirmations is OK. Your ATM Card Numbers is : 4001010257172299, Your Secret PIN Number is 8342. The ATM Card Value is $3.5M USD Only. Respond immediately with your full name, as you want them on the card and your delivery address/Phone Number.

In case you are wondering why or from where, it is a special grant from the World Bank, World Health Organisation, African Union, European Union plus other International bodies in conjunction, to all scam victims and towards alleviating the high level of continous rising level of poverty globally.

Congratulations once again

Dr. Felix Wilson,
Director ATM Unit
Scotiabank

Think, for example, of what kind of good this poverty fighting team could have done, instead of bailing out the Banksters of Wall Street, in places where people make less than a dollar a day. Worldwide - Malaria - cured. Social diseases, reduced. Hunger, solved. Education, improved.

Ah, well. Back to banging the rocks together this morning.

Xkcd adds shell mode

2010-04-01T08:03:00.000-07:00

Point and click (drool) interfaces do bother me. They are good for people that use computers, but not so good for those that need to communicate with them. Being able to type, memorize commands, and be able to combine them into short programs that do what you need makes the curve of learning how to program less steep - and better typing skills improves your ability to communicate rapidly.

Every time you have to reach for a mouse, you lose productivity. Ratons (the spanish word for mouse) es malo!

With the extensively mouse driven web browsing interfaces, we've raised another generation of button pushers.

That's why I was so overjoyed to see that xkcd added support for a shell mode this morning. You can type "next", or "prev", or (my favorite) "random", and get a new comic to brighten your day.

Can a renaissance of typists be far behind?

In search of Eldorado

2010-03-12T20:13:00.000-08:00

I keep meaning to write the words down on Mauro's new song... this is a reminder for me... and a listening experience for you.

Zoneminder/Facebook gateway and other random project notes

2010-03-02T04:44:00.000-08:00

I woke up with a really good idea this morning that I need to write down but first an aside.

I am very happy that facebook supports jabber now (XMPP). That makes it the LAST of the chat protocols that I have to wedge into one multi-chat client (pidgin or emacs' erc).

( That is, if I ever get bitlbee 1.2.4 + otr + bitlbee skype plugin + znc working again. My last attempt at getting otr patched into bitlbee failed big time. )

I really hate how "chat" - merely talking to another person over the internet - has fragmented into 14+ major protocols including irc - which has itself, fragmented into 260+ irc networks. Perhaps it's an example of the human condition that we form tribes in this way, that the tower of babel lives on... (has anyone rigged a translation bot into chat with google translate yet?)

Anyway, that's the rant part of this morning, on to the interesting stuff.

One major subproject of mine has been trying to come up with a working security and sensor system for my home. The hardware side of the software I've been calling the "pocobelle" project, and part of the software side is built around zoneminder, which is a motion detection/video capture/alert system.

A month or so ago, I got zoneminder to automagically push alerts ("Motion detected at the front door, see http://whatever/whatever for details") into jabber. It was just a couple libraries and a few dozen lines of perl to do this. (cool)

To make it faster and more reliable for local alerting, I installed my own jabber server on the pocobelle box... and also tried to register a zoneminder specific account on the main jabber.org server.

Unfortunately jabber.org hasn't been taking new signups, which frustrated me because I'm trying NOT to use google (which also does jabber) for new stuff and wanted to demo the idea to a few people - and because the zoneminder box is behind NAT I can't use the nifty federation (x2x) feature that jabber has which I could use to "link" my jabber server to another jabber server like jabber.org - AND because although jabber works great over IPv6 nobody seems to do x2x over it....

I woke up this morning with a new idea. Since facebook supports jabber now, I could setup a jabber account on IT, and have zoneminder send it's alerts out via facebook. Not only that, but by using the facebook API/uploader I could have it automatically upload the captured video, and using the facebook friends system only allow certain people to be able to view it...

That's it for the new ideas on the zoneminder front. In other news... My mom came down last month and brought 4 of Ubiquity's nanostation M5 5Ghz Wireless-n capable radios. I just got them configured - 2 with the default firmware, 1 with openwrt built by openwrt and (this weekend) my own openwrt build...

Pluses - The hardware is great, it's low power, and rated to 70C, and it has a nifty POE passthrough feature for the second ethernet port. The factory firmware is dirt simple. The web based site survey tools are great. The power monitoring LEDs are useful. I can get two of the radios to talk at 108 megabits/sec, with a real transfer rate in the 80Mbit range. I'm told I can do better than that at longer ranges than within my house (shorter ranges overdrive the local receiver), but it's still basically double the theoretical performance of wireless-g and easily triple the real world performance I've observed with wireless-g...

Minuses - Lacking IPv6 and (especially) routing protocol support, the factory firmware is USELESS. Maybe I'm spoilt by dd-wrt and openwrt having support for ospf and olsr out of the box, but believe me, as soon as you add a couple routers to your network, having a routing protocol to automagically figure out how to get stuff from point A to points B,C,D is flat out necessary... and it's not there. (aside: the regular unavailability and lack of standardization of routing protocols is what made bridging so inevitable - there is ONE standard for bridging - stp - and everybody implements it - and it works well for most things)

Secondly the factory firmware is based on a ancient version of linux - 2.6.15 - and that in itself, scares me. That's over 4 years old.

So I installed openwrt, not with a little trepidation (you need to install the nano-m firmware via tftp from a recent build of openwrt trunk, then install a bunch of modules like ath9k, and yes, I'll write up more documentation)

oooh, luxury. 8MB of flash and 32MB of ram goes a LOT further than you'd think. Linux 2.6.32.9, even, less than 4 months behind the mainline linux kernel. I installed ipv6, babel, ahcpd, quagga, snmpd, ntp, and a few other things, and still had room to spare. The web interface is not all that great, but I don't really care, I want to manage these things via command line tools anyway. After some tweaking, I got openwrt talking to the ubuquity firmware...

but so far, only at 54Mbit. I'm missing some configuration parameter (I hope). I have a 40Mhz channel configured, but... I'd like it very much if I could basically drive the radios at 100Mbit ethernet speeds, it would lower the need to shape the outgoing traffic as much, and anytime I can get something that is 5x better than an older technology (wireless 11b) I'm happier...

I also haven't figured out how to override the power settings to the full range available, limited to 17dbm of gain instead of 27dbm...

Update: These two problems are related. I'm definitely overdriving the receivers. I can actually connect as high as 162 Mbits/sec between the two. I found this out when I accidentally pointed one of the radios at the ceiling... Ooh. 162Mbits.... NICE...

Now, since I envision a day where I have to build out a bunch more of these radios, and partially because I'm a glutton for punishment, I downloaded openwrt and built my own firmware. That worked... on the first try! (that's amazing in itself, but usually a bad sign indicating real trouble ahead, as I will always then proceed to dig my own hole). On my second try - after configuring a few more modules - it fails on building the kernel for some reason:

LD [M] fs/autofs4/autofs4.o
mips-openwrt-linux-uclibc-ld: unrecognized option '-Wl,-rpath,/usr/local/lib'
mips-openwrt-linux-uclibc-ld: use the --help option for usage information

I'm stuck on that right now. I have a bunch of tests to do (reliability, traffic shaping, different routing protocols, etc) but I'd like the two different firmwares to at least be talking at the same rate.

But it is kind of neat to see babel "just working" to route between the routers. Hmm... maybe I can get vlans to work... I wonder if this thing has any pins I could hang a temp sensor off off...

Rebooting for 2010

2010-02-12T07:46:00.000-08:00

Computationally, I've limping along, flying on the shreds of one wing. My quad core box died in August, my laptop died in December, and I killed Pocobelle back in November.

The quad-core? Dead beyond repair. I tried replacing the fan, the cpu, and the memory. Pocobelle requires some finicky jtag work to repair and was at the end of it's useful life anyway, so I replaced it with an open-rd box.

And, rather than fix the laptop, I ended up building a new machine (specific to a project I can't talk about), in x86 mode. "Buddy" is a nice machine, a dual core atom, and the zotac nvidia graphics interface is amazingly fast, even with driving two screens. I don't miss my old power sucking quad core box, except when compiling kernels and ardour. I should have made buddy be x86_64, however, so I could just clone the laptop...

Back in august, I switched over to using emacs for chat, email, news. I took major steps to try and integrate chat, in particular, into emacs - using bitlebee and znc to integrate skype, irc, and jabber into one interface. It worked great!

I do the majority of my writing and coding in emacs, already, so I ended up with two fullscreen windows that used every pixel I had available, sparingly.

I wasn't done, in particular I started working on some new blogging software and got stuck on it, and the machine I was developing it on, pocobelle, died also, and the memory stick it ran on is around here... Somewhere...

It took a LOT of work to setup all that I haven't replicated on the new box. It would have been easy had I stuck with x86_64 mode instead of reverting to x86. Instead I switched to using the new thunderbird for mail, dropped netnews entirely, and went back to pidgin for chat, because it was easier to set those up quickly. And went from the keyboard driven window manager I like, back to gnome. A major mistake, I'm thinking all that was, but I did get a chance to try some new tools.

Thunderbird 3 is awesome, in particular the tabbed searching facility is to die for. But I haven't got around to sorting my mailboxes with the same level of filtering I had with emacs. Due to switching to using imap for the email backend, I'm not sure how hard it's going to be to use emacs for mail again, and I'd rather like to keep thunderbird around as an option.

I HATE the gnome window manager. It's so dumbed down as to make me feel like Harrison Bergeron.

Pidgin is good but...

The thing is, I can feel the productivity seeping out of me every time I switch from one glossy white (web,chat,news) window to another. I feel far less productive minute by minute, hour by hour, by using these tools not integrated into my main editing interface. My personal, searchable database of everything I do was all living in emacs, and it was letting me manage far more "stuff" than I can without these tools integrated.

Sigh. So I kind of need to reboot my life - repairing the dead laptop, resurrecting pocobelle, getting a new desk made, laying out all the stuff that needs repair or upgrades on a workbench...
and some time all by myself to methodically go through it all.

Facebook has finally adopted a standards based chat system, built around XMPP. That's one timesump I can just move to being chat. There's also a fix for yahoo out there in bitlbee. My first attempt at integrating the yahoo and otr patches into bitlebee did not go well, I hope someone else has fixed it.

I built out my email systems using digital certificates. They are about to expire and the cert I had for the website to un-expire them is on the dead laptop....

Hopefully I'll find the time this month to rebuild my environment to where I feel productive again. What I really want to be able to do is wake up, and sit down, and work, without having to reboot my world every time I get up. Solving THAT problem is going to require I get a solar/battery/inverter system so that I can keep the core systems powered up, and/or that I simplify my environment a lot more... need money for the inverter system...

Maybe I can outsource some of the backlog. Need money for that. Guess I need to work some, first.

While all that stuff was going backwards I did make major forward progress with "jaco", the open-rd replacement for pocobelle. It's a darn nice box.

At one point I had high hopes I could actually use jaco for a desktop replacement. At 11 watts, it eats ~1/4th the power of the atom, and runs emacs just fine. Given that my display eats 20watts, I could halve my power consumption and double run time on battery. (Lest you think this is an obscure requirement, I frequently undergo day long power failures, and running the generator for long periods is both annoying and expensive, and my existing UPS lasts about 3 hours on the atom)

The video driver, however, is so horribly slow that I just can't stand using it as a desktop.

Jaco is great for everything else though. I have it running zoneminder, in particular, as part of my home security system, in addition to it being primary DNS, squid cache, web server, bittorrent client, ipv6 gateway, mesh networking, email/imap, znc, jabber server, music and I forget what else.

I have had so much fun developing stuff to work on the open-rd that I've been afraid to turn it to the production use I'd planned for it, namely replacing my router/firewall. I guess I need to get another one, or wait for the new "guruplug"s to arrive.

First up this morning - make an outline of everything I need to do this month. Importantly, this has to include revenue generating stuff, as I burned myself out back in December and need to get on the stick, but also importantly it has to get me to where I can "wake up and work"....

Second up this morning - fix the !@#@! Laptop. To do that, I need to find the pesky little screws for a new DVD drive, and burn a new cd, and that new DVD drive is going into the box I'm typing this on so I guess I'll have to power off in a sec before I can develop more of an outl

The Shuttle launch this morning

2010-02-06T23:41:00.000-08:00

I started re-reading Russian Spring this morning, at around midnight, so I could maybe get to sleep.

I completely, utterly, totally forgot the plot, and the characters... and it was once one of my favorite books! This has happened to me a lot in the last 8 years, and it's still hard getting used to.

Whenever I get under the least amount of stress these days I have memory problems but to lose an entire book is unusual. I'm under stress, my mom is visiting, and a good buddy is getting married at my place next week... All day I was aphasic, losing and/or substituting words.

All I remember about the book was that I cried several times, reading it. Page flipping around it now, I can see I will have a lot of empathy for the main character, Jerry Reed.

I had intended to give the book to a friend yesterday, but forgot to do that.

I can tell that Russian Spring was on my mind, at least in part, due to some of the setting being South America, and the recent cancellation of the Constellation project...

... and since I'm so messed up, I'm also listening into and watching via NASA TV the upcoming Shuttle launch of mission STS-130.

The video and audio keep going out. I wish there was a way to turn off the visual feed, and just get the audio stream. When I was a kid my dad used to mute the TV announcer of the baseball games, and tune in the radio, the games were much more vivid that way. Aha! there is a real-audio audio-only stream... that's working good. I wonder if I can get a video only stream.... oh, yes, just mute the audio on the video stream, that's working, intermittently.

Waiting for a spacecraft to launch is one of the most boring, yet interesting things there is. As hard as the announcers try to point out ever detail for future generations, how getting to space was done, this morning, feb 7.

And still my heart leaps to my throat when I think of the candles being lit.

You just have to keep remembering.. that in all that boring detail is that several peoples lives and a billion dollar payload all ride on the intricate dance of thousands of people...

Having a good book handy, helps too.

It's T-42 minutes, and counting. The launch is scheduled for 4:39 AM EST, but the weather is steadily getting worse, with only a 30% chance of launch.

I think I'll just listen to the launch. After all these years I still can't forget the images burned into my mind that I sing about so often, but I have forgotten everything after the 3rd stanza of Rhysling and Me, entirely, substituting a flute or guitar solo for the sad part and a new ending. I've played that song thousands of times now, and burned the memory out of me. Even clicking on that link, the forgotten lyrics enter my brain and exit immediately. For forgetting those, I'm grateful.

... some time goes by. It's at a planned hold at T-20 now...

I check the temperature - the announcers haven't mentioned it - according to it's 48F, 8.88889C. Superstitious of me, I know...

... T-9, and holding...

I'm re-reading Russian Spring, slowly, my attention split between the radio and the book, now, after thumbing through it earlier this morning looking for a passage I recognized. Oh, yea! Page 10 I remember - the chocolate ice cream Jack Reed had, at the age of 4, in the early morning of july 16th, 1969. I'm the same age as the lead character, but I don't remember what I ate that day.
My earliest memory of the space program dates back to probably 1971, because I remember how cool the moon rovers were... seeing them shoot lunar dust straight up and having it fall so oddly in the 1/6th gravity.

I could go for some ice cream right now. Don't have anything sweet in the house, actually, except a coke. I'll have to settle for that.

"It's an experiment, Jerry" Uncle Rob told him. "The greatest moment in human history is about to happen and you're alive to see it, but you are too young to remember it without understanding. So what your dad and I are trying to do is implant a sensory engram in your long term memory so when you grow up you can call it up and be here now with your adult conciousness"

That "greatest moment" was over 40 years ago.

In the interim, we've managed to make spaceflight boring. We went to the moon, and found nothing worth using. We retreated to near earth orbit, constructing several space stations, concluding with the ISS, to which this is the 29th payload. We directed probe after to probe to the rest of the solar systems, but nothing since then compares to those moments when we first strode upon another planet.

I've had people from later generations, intelligent people, educated! people ask me if I thought the moon landings were faked, and I've been to dozens of space conferences, shaken Buzz Aldrins hand with tears welling in my eyes, seen the launch of Spaceship One, and it's all I can do to stop myself from doing what Buzz did to one persistent doubter... I restrain myself, relax, and patiently explain, that:

"That yes, it really happened. Yes, men, Americans all, really did, once upon a time, walk on the moon".

After I'm gone, who will remember, who will still believe?

... There are 11 minutes remaining in the hold. The weather remains a problem... the pace of the operations team increases, the tension rises, but Flight is "no-go" for weather. He's saying that "I think this is not our day today" and after a brief conference with the other controllers, Flight decides at 4:30AM EST:

"Let's go ahead and knock it off."

He calls the shuttle:

"OK, Zambo, we tried really hard to work the weather, it's just too dynamic ... and we're not comfortable launching a shuttle tonight."

Endevour copies:

"We thank you all for the effort you put into night. Sometimes you got to just make the call."

You can hear the heavy disappointment in the flight controller's voice, as he takes a deep breath and makes the hard decision, bringing the rest of the team up to date.

"Let's scrub this launch attempt, and try again tomorrow night".

The rest of the team kicks into scrub mode and starts making the vehicle safe to unload. It's hard to contain my disappointment, but we can't control the weather, yet, or build a rocket ship that can punch through it like an airliner.

Tomorrow, maybe, a shuttle will launch, although I hope to be safely slumbering and just read about it tuesday morning.

...

I've wanted to type up my reaction to Constellation's cancellation since Monday. The reaction from newspace is generally positive, Jon Goff's reaction mirrors mine, but I do have a lot more to say on the subject that I'll say once I can get my thoughts together.

I think I'll try to read myself to sleep. I'm on page 12 of Russian Spring now, and remembering more why I emphasized so much with the main character. I'm not looking forward to being made to cry again.

The problem with economic coercion

2010-02-02T07:35:00.000-08:00

MAN: Will you sleep with me for a million dollars?

WOMAN: Are you trying to set up that old chestnut?

MAN: No, it’s a serious offer. [Shows briefcase full of money.] I’m very rich, you’re very beautiful, and I’m willing to pay you a million dollars for a night of glorious sex.

WOMAN: Rape! Rape! Rape!

MAN: What?

WOMAN: Rape! You’re using your million dollars to force me to have sex with you!

MAN: What? All you have to do is say ‘No.’

WOMAN: You don’t understand, ’cause you’re rich. It’s a million dollars! It’s an offer I can’t refuse! It’s economic coercion being used to force me to have sex with you. Rape! Rape! Rape!

Deep Lurker

ESR's blog

There's quite a few things about this modification of an old joke that seem profound. We all have a tendency to do things for money that we otherwise wouldn't do. In my case, despite trying for many years now to spend as little time as possible in the programming world, it's still my main bread (and margarine). Programming messes me up - I have to think in English (although I have been somewhat successful in thinking in Spanish while programming on my own projects, working on other projects requires thinking in the standard language), which affects my ability to communicate down here. I get seriously OCD; Nerdy. My relationships suffer. I have a hard time sleeping as I can't turn my mind off. My back hurts, as do my fingers, and while I am deep in a project, I have a hard time fitting in other things that are good for my health. I put myself through death marches...

I have a list of 20+ other sorts of work I can do that don't mess me up, but none of them can bring enough revenue to keep a roof over my head. Do I cry, "Rape!"?. No, but the joke does have a point. I THINK that I can handle 6-8 months a year of it, spread out some, but really have to watch myself lest the computer take over my brain entirely, during the project. Living where I do now, I have that choice available to me. Were I still living in the USA, I'd be back inside a 70 hr week, 50 week a year grind, and still only barely making it. I joke now, that I used to work 18 months out of the year.

I remember a character in a Neal Stephenson book, who had "Poor Impulse Control" tattooed on his forhead - it didn't help. I have a friend, with "Never Again", tattooed on his chest, in reverse, so he can see it in the mirror every morning. That doesn't help, either.

It would be nice to live in a world where you could work at pretty much anything, and get by, and never have ethical or physical conflicts, and do what is right for you, long term, all the time. I don't think such a world has ever existed, except for people that worked hard at identifying what was good and what was bad, for them, and made the difficult choice when faced with a suitcase full of money. And they needed to have the time to contemplate their ethics, long before the suitcase entered the picture.

Sort of related to this is that earlier this year I was buying and drinking soda by the case. I couldn't control my intake, so I just stopped buying it, switched to water, and lost 5lbs in the weeks afterwards. I know that if I still lived in town I'd buy it a lot more often, but living far from civilization means that I only have to exert self control - very consciously - twice a month, when I go to the store. I work off my list, and just my list. I think a lot of people - including myself - can't cope with or function within the minute by minute temptations of the modern world.

Side note: Recently I got interested in the somewhat perverse history of Stevia, a natural sugar substitute. I also learned (actually, I'm not convinced this is true, yet) that due to the amount of sodium in most sodas - that you need to drink more in order to meet your water uptake needs! The "Pause that refreshes", actually, doesn't.

I think I'm going to learn how to grow Stevia(if it's legal here) and make my own sodium-free and sugar-free soda, and see what happens, or maybe try a cane extract.

Does the ready availability of nearly-narcotic substances like sugar, coupled with sodium, amount to "Rape! Rape!" as in the joke above, or do I need another word?

Late last year, there was a huge thread on my blog about the real issues of health care. I'm still working through the comments on it (one of my regular commenters, cpm, was on a roll!), and would certainly like more people to weigh in.

Some of that kind of comes down to this sort of economic coercion - where bad foods like fritos cost less than good foods like vegetables - or, where your taxes are used to pay for other people's problems, so why not abuse the system yourself? - or being constantly subject to advertising that tries mightily to convince you you will be sexier, more handsome, richer if only you'll buy the product being advertised....

Also, in the blog entry, I recalled a story I'd written years before about a dystopic future where it made more economic sense for a young and healthy individual to sell off their extra organs rather than find a real job. Economic Coercion? Rape?

I'm REALLY bad at handling advertising, and run with as many as filters on my internet connection as I can so I never even see them. While this too, helps, it leaves me vulnerable whenever I leave my safely cocooned connection for someone elses - I do make it my mission to install adblock plus on every other friends computer I touch. It's not altruistic. It's self defense. Should I cry RAPE!?

Ah, well, I think, after writing this, I'm going to go do a few laps. At a basic level you still can choose to do what is right for you, for many things.

Happy New Year

2009-12-31T10:50:00.000-08:00

My mom just finished this painting of Josh Täht. Nobody I know has seen him in a year or two. He's still somewhere on the streets of New York, so far as I know.

So, anyway, I do wish a happy new year to everyone. I hope that this year will be better and different for everyone, especially Josh.

It's time to wash out the old year, and bring on the new, and have fun, and flirt, and dance, and play!

But I think I'm going to play "Redemption Song" tonight, slowly, and sadly...

and then I should be able to rock out.

Rad Decision

2009-12-01T17:15:00.000-08:00

Tonight's online reading was Rad Decision, a novel about the events leading up to a nuclear accident in the US that didn't happen, but could have.

I found the the plot gripping, the situations utterly believable, the characters decent, the backstory quite plausible, and the denouement worth thinking about. Recomended.

I still think nuclear technology belongs in the mix of energy sources for the future.

I'm impressed with how far Nanosolar has come in the past few years, although I find it weird that they are concentrating on Germany rather than in places, like Central America and Mexico, that have boatloads of solar power.

I do appreciate the rush towards greener consumer technology, LED lighting in particular, seems to be getting better at a rapid rate. Earlier today I'd given up on getting any sheeva plugs this year, and went with an open-rd box, which promises about 1/3 the power consumption of a nearly equivalent PC. I also decided not to build out another quad core machine to replace my broken one.

Whether or not the open-rds could be used as a desktop and be as useful as an atom (the nearest competitor for the desktop) remains to be seen, but I intend to retire one of my computers in favor of it for use as the first iteration of Pocobelle 2, probably with added duties as a music server, and perhaps it - or the next Arm A9 based generation - will give intel a run for their money.

I read David Rowe's blog religeously. Here he talks about the power audit he did on his house and what he did to improve his negawatts. He also talks about what he did for his pool system to improve it, too.

I'm also big on geothermal power - I've been watching a Nicaraguan company, formerly known as Polaris Geothermal, now known as Ram Power complete a merger and obtain 150+ million in funding to bring up (at least) 70MW more geothermal power here. A terawatt is feasible, long term.

Nicaragua has enough geothermal energy in the ground to power 3 countries, if only they'd - or someone - would try, harder.

Through such slow, incremental change is the future made.

I'd had a really bad week last week. Nothing for me, personally, was really working, and I'd had some bad news about the Montavista merger that basically tore up some retirement plans entirely.

I took a cheap cab out halfway to Rivas, as I was too broke to get all the way to the frontier to renew my passport and return - and started walking south. I made it as far as the windmill project down that way, which has been under construction ever since I'd got here, with numerous hassles and delays.

The wind off the lake was blowing hard, and every last one of those enormous shiny white windmills was turning, and they'd finished running the power lines out - They were actually delivering power!

It was so beautiful that I sat down in front of the land, watched those windmills turn and turn, and cried. Maybe only another engineer can understand what it is like to struggle and fail and flail and fail again and then see something else that was actually working.

Shortly thereafter someone picked me up and took me to the border.

I made it home with 70 cordoba in my pocket, but my good friends at El Pozo let me run a tab, and all the friends I hadn't seen in a month showed up that night to visit.

So my life goes on, with a few moments like these to make it worthwhile to struggle on, and to keep trying. I keep saying to myself that the only way to win is to not play the game, but it's deeper than that, you need to invent new rules for the game. Better ones.

Still not finishing the spec for Pocobelle 2, but satisfied all the same

2009-10-25T11:40:00.000-07:00

The big reason why I haven't been participating in the otherwise fascinating thread on health care on my blog is that I swore to myself I'd finish the spec for Pocobelle version 2 by this weekend, and get it posted, and commented on.

Hopefully, with some feedback, I could order the "right thing" by the end of next week.

I write stuff and post it so that - although the average intelligence of the internet may be low - the cumulative intelligence is also inconceivably high. Out there there is always someone that has the answer to your problem...

Examples of cumulative intelligence are the health care thread, and what I read today on slashdot about embedded hardware while taking a mental health break.

I had pretty much settled on the TS7500 board as being pretty much ideal for what I was looking to build. It was extremely low power, and had enough DIOs, host USB, flash, etc to do most of what I needed to do.

The only problem with it was that it didn't have enough ram to do all I needed to do, and was kind of slow. I was working out the implications for motion detection in zoneminder, trying to offload that function to other devices, and not liking what the shifts in costs were like - my camera costs went way, way up, and functionality down. Zoneminder's motion detect algorithm was very, very good, but also cpu intensive. I couldn't see my way clear to embedding anything even as close to as good in a FPGA, and doubted your typical camera could, either... and I was trying to keep the moving parts and power budget to a bare minimum.

Slashdot steered me at two really cool looking products - most notably the Sheeva Plug, and secondarily the Open RD...

Which - aside from slightly higher power consumption - still far less than a laptop or atom board - are pretty much perfect for most of what I need (except the DIO relays, which I can probably do via USB).

I kind of wish they'd posted on this topic before I burned the hours writing my spec, and for all I know there is some showstopper in one of these boards, but it hurts a lot less to read than to write, so I'm going to go read up on them now, and maybe place an order or two.

The sheeva plugs have 512MB of RAM, 512MB of flash, USB 2.0, eat 3w of power at idle, and cost 99 BUCKS! for that price someone can afford to buy a couple, just to play with!

I am sore tempted to order a a touchbook, too. Life is getting a bit more interesting in the arm world.

taking a brief break from health care for something more geeky

2009-10-23T20:59:00.000-07:00

My last post gathered a lot of great comments and the debate continues, hot and heavy, on the subject of health care.

I surfaced for a little while, and spotted something that appealed to my ha ha only serious and geeky nature... if you don't know who's in the photograph, you won't get the joke.

OK, I return now to some voluminous commenting on health care.

Reframing the issues of the health care "debate"

2009-10-21T15:30:00.000-07:00

I am despondent on the whole health care reform thing.

I view it as a circus distracting people from the real issues, actually.

I was nearly driven bankrupt, twice, by the current system, and have a long list of detailed complaints about what is wrong with it at present. Coping with the current system, as it exists, with a pill for every malady, and pills to cure the side effects of those pills, and pills to cure the problems caused by the other pills... made me considerably more sick than I needed to be, for far longer than I needed to be, and coping with the stresses of the system didn't help either.

However, living where I do now, I don't have to pay into the US system any more, and I don't intend to go back. In fact, the risks involved in going back are so high - one traffic accident, and I would be on the brink of bankruptcy again - that it is a profound disincentive to even think about visiting the USA again.

So I really shouldn't care, anymore, but I got into a debate about it the other night and thought I'd type up my side of the argument.

I think that the legions of non-medically trained auditors and lawyers and lobbyists should not exist.

I also find the idea of one nationwide universal health bureaucracy for 300 million people abhorrent. I believe that the central government of the US should be as weak as possible and mostly concerned with national defense. If a given one of the 50 states wants Obama-like health care, great! Maybe people will want to live there more.

For example, both Massachusetts and Tennessee implemented Obama-like care a few years ago.

How's it working out? Is it a success? How are the demographics changing?

It is far easier to experiment with public policy at the state level than it is to do so at the Washington, DC, level, and far, far easier to rectify mistakes. Even then (witness the decline of California vs NJ, for example) it may take decades to reverse course.

Most importantly, most of the developed world is in a demographic trap. The old are getting older; there are not enough of the young. The haves - mostly the boomer generation, now - is trying to extract as much money as possible from the have nots - the successor generations - at every step of their slow decline into senility, so their Woodstock goes on and on while we have to clean up the messes they've left behind.

There are easy solutions to the demographic problem - lots and lots of immigration, easy citizenship, and incentives for child-rearing - that aren't on the table, and won't be until it is too late.

I wrote a story once, called "the good life", which was about a life insurance salesman that sold the "good life" to those young and healthy - cars, booze, parties, trips, etc - in exchange for a few organ donations when they hit 30... And naturally, being raised ignorant and away from valid work, they'd go into debt selling off more organs and limbs, partying further, after that, until they had nothing left but a brain attached to a few machines, and big payments to make on keeping those running too, and after that... nothing.

Their organs and limbs went to the old, who were living on cruise ships....

The old preyed on the young in this brutal dystopia until there were no more young left to support them, and revolution ignited between the oldsters fighting over the last "useful" young people, who by then were being bred in jars and kept in the matrix, dreaming, until they could be harvested for spare parts.

...

Jerry Pournelle writes:

"... the one fix we know will not happen is some modification of the malpractice laws and the tort system. That simply will not happen. There are several other sacred cows that will not be touched no matter what changes are made in the system. That's simple politics. So: given that some of the obvious fixes are impossible, and will not happen, is the system so broke that it needs a fundamental revision that will not make the obvious fixes like tort reform? ...

And I ask, again, the obvious: where did your obligation to pay for someone else's health care come from? Under what principle is this to be done? The pragmatic argument is that it will save money if everyone is put under a health care plan, and preventive medicine will save huge
amounts. I doubt that those who say this believe it; certainly few others do. We don't need health care reform to put a tax on sugar pop or even to end the sugar protections and subsidies; is that likely? Is there any evidence that preventive medicine applied to an entire population lowers health care costs? Depends, of course, on the preventions: cleaning up toxic wells and providing clean water certainly works. Does exhorting people to lose weight do it? If so, on which audience do the exhortations work?"

I have a long list of things that would help American health that are politically unacceptable - from banning sugar and corn subsidies (as per the above), to taxing fast food restaurants, and taxing or banning bad foods, full of carbohydrates and such (and subsidizing vegetables in return), to taxing tivos, tvs, and game consoles (and in turn subsidizing gyms, yoga retreats, and international travel)

Here's another one: Why on earth do businesses still work 9 to 5, with employees wilting under dim florescent lighting, when outside there's a sun already shining? Is it to sell more vitamin D cream, gym memberships, and anti-depressant medications?

None of these solutions are on the table. On my bad days I tend to think that modern society is oriented towards killing off those that cannot summon the will to turn off their tivos and take a walk once in a while.

Lastly I'm going to quote a few more choice words from Jerry Pournelle on the issue of the largest health care cost of all - treatment near the end of life:

"one reason health care costs so much is this business of the last two years of life. If we could eliminate those two years we would no longer be spending more than others. Or eliminate even one of them.

And that poses real moral dilemmas. We know how to save the money. Have the good grace to die. MIT economist Lester Thurow never put it that bluntly, but he's been talking about this for a least a dozen years, as for instance in the lecture I attended in Boston as part of a AAAS meeting about that long ago when he gave a very good presentation of the dilemma of health care planners, and as part of it talked about the Esquimaux culture's solution: "It's a good day to die." Of course I was considerably younger in those days.

The moral dilemma comes when you try to plan out who shall make these end of life care decisions? And who shall pay for them? If the dying person has the resources to pay for all that expensive treatment, should the state prevent that "waste", so the money can be inherited and subject to inheritance tax? And who shall have the right to speak for the dying person? Who can determine whether that person is mentally competent? But one assumes that the moral principle is clear, the property owner has the right to use that property to preserve every last
second of life. Note that even with that clear principle the details get sticky.

Now suppose Grandma doesn't have the resources? Or, if they are used to keep Grandma alive with tubes, all the family resources will be gone, and Grandpa will be on the dole? It's still their property, so that's just another complexity.

Now suppose they don't have the resources at all. Someone else must pay for both their end-of-life expenses. Who is obligated to make that payment? Is it an entitlement for all and an obligation on the public purse (and which public purse, local, state, or federal)? If it's an entitlement for all, then these costs are actually the property of the patient, and the individual decides whether to spend great sums on staying alive.

Of course there are details to worry about. Clearly we can find some cases in which we'd all agree that prolonging life at great expense just isn't worth the money it's costing us. I can also cite cases of 96 year old grandmothers coming out of quadruple by-pass heart surgery with a
couple of good years (possibly more) of life ahead. Now what?

These are the questions that need answering when we evaluate the cost/effectiveness of health care. I don't think they are actually being debated in Congress, because the political implications are so great.

More another time: but that's why health care costs so much. So very much of it is spent in the last two years of life. The cost reduction is obvious. What is the effect of that saving, and who decides?"

In anticipation of the flames, it's probably a good idea to put on my asbestos suit now... even though that causes cancer, in the long term.

20 years ago tomorrow

2009-10-17T15:22:00.000-07:00

There was an earthquake that destroyed large portions of the town that I still think of as "home", Santa Cruz, california. I wasn't there, but when I moved there 6 months later, everyone was still kind of traumatised. People coped in various ways. Everyone there on that day has a very clear memory of it, and in some ways I wish I had been there for it.

My friend Evan - who I hadn't met yet - coped with the quake in his own distinct way:

One mistake, and PocoBelle becomes a brick

2009-10-11T08:52:00.000-07:00

It was a dark and stormy saturday night. Power and internet were offline. I'd got to where I was satisfied with the Linux kernel I was running. It was doing everything I needed to do, I'd loaded up all my apps and (with a 256MB of swap) hadn't had a crash in a week...

I wanted to focus on userspace, and make Pocobelle boot standalone so it didn't need to fetch, via tftp, its kernel anymore, and could stand completely alone...

(About the only bit of tuning I wanted to do was make it boot faster, I had had to put a rootdelay=10 seconds into the Linux boot so I didn't have to do any special magic (e.g. switchroot) to complete the boot off of the USB stick. But it booted in less than 2 minutes, and that was fine.)

I was *happy*. (Getting to where you have a stable kernel makes EVERYTHING else a thousand times easier)

Soooo... I decided to write the the "stable" kernel to the on-board flash. I went into RedBoot and did a:



Fis list

Name              FLASH addr  Mem addr    Length      Entry point
(reserved)        0x60000000  0x60000000  0x07D04000  0x00000000
RedBoot           0x67D04000  0x67D04000  0x00040000  0x00000000
vmlinux           0x67D44000  0x00218000  0x00160000  0x00218000
RedBoot config    0x67FF8000  0x67FF8000  0x00001000  0x00000000
FIS directory     0x67FFC000  0x67FFC000  0x00004000  0x00000000

Not remembering how Pocobelle got this way, (I'd had it in a box for 4 years or so) I blithely assumed that this was all I needed. I kept scratching my head over how this layout failed to match what was in the kernel for the layout:


static struct mtd_partition partition_info128[] = {
 {
  .name  = "TS-BOOTROM",
  .offset  = 0x00000000,
  .size  = 0x00020000,
 }, {
  .name  = "Linux",
  .offset  = 0x00020000,
  .size  = 0x07d00000,
 }, {
  .name  = "RedBoot",
  .offset  = 0x07d20000,
  .size  = 0x002fc000,
 },
};

My assumption was basically that I had an advanced kernel with a preproduction redboot and that the partition table info here was incorrect...

WRONG.

I did an fis init -y to clear out the uselessly small Linux partition, hacked the TS7250 driver to take my new partition table, built a new kernel,

Allocated 4MB for the new Linux kernel at 0x60000000. Erased that partition, tftp loaded and wrote the Linux kernel there...

I completely forgot that this device has a complex bootstrap process. The tiny on-board NOR flash has a tiny bootloader that then bootstraps another bootloader out on the NAND flash, which is then smart enough to load up Redboot, which in turn is smart enough to load up Linux.

Result: 1 brick. Pocobelle never gets to Redboot. It jumps right to Linux and blows up shortly thereafter.

I figure I overwrote the TS-BOOTROM, which, although it wasn't marked in the bogus fis table, was hiding at 0x06000000. The system has enough brains to jump to that spot, that's all....

Now, there may yet be something to do that will get me into rewriting the ts-bootloader into the NAND flash from the serial port. But I doubt it. (I'm writing this blog entry unattached from the internet, like I do a lot these days, so I can't google)

Post googling Update: Joy! There's a tool to boot from the serial port! Boo! I have to build redboot from scratch for it to work. I've been meaning to do that anyway, but...

There's no one to blame for this but myself. I should have dumped the contents of the flash there and looked at the strings in it to see what it meant before I overwrote it. Shoulda, but I didn't. I normally would have done that, back when I was doing this professionally. I would have puzzled and agonized for days over the mismatch between the flash info, the documentation, and the source code...

(Back then, the boards I was working on were usually very early prototypes, worth thousands or tens of thousands of dollars. My fear factor with a 145 dollar board is considerably less.)

It would be easy to fix if I had a jtag interface available, or was still living in the Silicon Valley, where I could visit any of a bunch of friends at companies that keep stuff like that lying around.

Sadly, this is the furthest I've been away from a jtag debugger in a decade. I keep meaning to get one, but they tend to be rather project specific and I've been unable to settle on the right boards for what I intend to be doing.

The nearest jtag debugger is probably 1200 miles away, in Florida, maybe further than that as Florida isn't exactly known as a hotbed of embedded development.

OK, so I rationalize to myself:

This was just an experiment, after all, to see what I could do that was new and different in the embedded world.

I've pretty much determined that the range of software I want to run is going to require at least 256MB of ram, and I might as well go for 512MB. I was kind of hoping to keep prototyping on the board (it was already quite useful as it was).

At the time that I killed Pocobelle... I was successfully running (all over ipv6):

Ipv6 tunnel

bind9, with views, dnssec, and ts-keys as my primary domain server

postfix as a mail router and spamkiller, with ca-cert certificates for security

pgp-enhanced-mailman

openssh

inn (yes, a full netnews server!)

cricket (for network monitoring)

youtube-dl for grabbing youtube offline

madplay for music

lighttpd and apache

My new, still in development, blogging software

rsync for backups

All on this 300mw power sipping machine. I'd transfered every function I'd had a dedicated server (an old laptop) doing onto this box and was ready to turn it off...

The only things that were annoyingly slow was heavy duty disk writes, (databases in particular) and the startup of interpreted programs was very slow, particularly in the web service side.

I had just switched (yesterday) from apache to using lighttpd - which was much faster than apache and, as a bonus, gave me high speed .flv streaming) and was working on getting fastcgi to work with the web interfaces for dimp, cricket, zoneminder, and my new blogging software.

And I killed it...

The darn thing was getting useful. I was actually getting dependent on it... it was routing my /48 ipv6 network, and running DNS and mail for the whole house and was serving up a bunch of mp3s and videos...

I'll miss you, pocobelle. I'll get you fixed as soon as I can, I promise.

The beauty of this particular project is that I can retreat, for a while, into booting into a qemu emulator of the arm processor in it. I never needed actual hardware to run it in the first place. It WAS essential I prove the board and kernel reliable and get a feel for it's performance. That's it.

All the binary code for it actually lives on a USB stick. I can just pop that stick over there and resume working.

(This is so much better than life in the old days, pre-usb sticks, pre USB, even... I have spent months in my life in a jtag debugger, just trying to get freshly designed and probably buggy board to run the first 50 instructions...)

So I slammed the USB sticks into another machine, made a copy, converted the result into qcow2 format, and booted up pocobelle virtually via qemu. The out of the box Linux kernel I have for that emulator is the Versatile variant of arm, which needed a bunch of modules, so I booted up another copy of the emulator that accessed the original versatile system image, and copied those over. I told inittab to use a slightly different serial port (ttyAMA0) and /etc/securetty (to let me login on that port)

And vPocobelle came to life once again! (I still need to figure out the tun interface to get the emulator on the net, however)

It's actually 3x faster to run out of the emulator in this case actually!! And the kernel I was using was fully baked! I was done. I didn't need to work on it anymore! And I'd intended to focus on userspace issues anyway and see what more memory did for me...

So (temporarily) losing the hardware is a setback, but only a minor one.

I really, really liked that it let me stay on the internet for a week without power. I'm writing this now, without power, or internet... (Note to self, get more gas for the generator monday)

I have no friggin idea how or when I'll get Pocobelle to boot again.

Maybe I'll find some california surfer dude visiting that can pick me up one in the Valley on the way down here....

It's probably cheaper to just get a new board...

I wonder what that will be?

A quick automounter rant

2009-10-07T08:54:00.000-07:00

A nice feature added to most Linux desktops in the last couple years, is the robust support of plugging almost any USB device into it and having it "just work". Printers, USB keys, Mp3 players, pretty much all "do the right thing" now. USB support in linux is comparable to Mac and Windows now...

... Including the flaws.

I don't understand all the magic that takes place behind the scenes to get a mp3 player to mount on /media. I understand it picks up the partition table name somehow, and gets an event...

But...

Why do we mount the devices read/write? And leave them mounted all the time? And require human
intervention to eject them? And have bad things happen if you don't do that? And (at least on the box I'm in front of) require you be logged in to the desktop to have it automount it in the first place?

Linux has had an automounter since the days of Apollo (Not the NASA version, the HP version), and for local devices it works pretty well. Automounters have never worked perfectly of course, particularly over failing networks...

It turned out pocobelle didn't do any of the magic that my desktop did, not mounting the drive at all, so I sat down and implemented automounts mostly the way I wanted them to work in the first place.

I added the following lines to /etc/auto.misc

sansa -fstype=vfat,rw,uid=1000,gid=1000,umask=022 :/dev/disk/by-id/usb-Rockbox_Internal_Storage_90000000000000000A4B4511712031AFD-0\:0-part1

sansa2 -fstype=vfat,rw,uid=1000,gid=1000,umask=022 :/dev/disk/by-id/usb-Rockbox_Internal_Storage_90000000000000000A4B4511712031AFD-0\:0-part2

(These are each one line. yes, my sansa runs Rockbox)

I created symlinks to the device in my public_html directory, so when it was plugged in I could stream media from it from any of the other devices in the house. When not plugged in, attempts to access it result in file not found, which is what I want. Also, when it is plugged in, I have a cron job setup to suck down some music at night from some internet radio stations I like (and not when it's not connected)

According to the setting in auto.master, it auto-dismounts 60 seconds after the last program using it stops, which makes it easy to just unplug it when I want to take it on the road... 99.99% of the time it will be dismounted - and I'd like it to mount read-only until something wants to write to it, which would take care of a few other nines... and implement a forced killall processes and umount triggered by a switch on one of the DIOs on the board to take care of the last .000001% of the problem.

Some news from the life and death and life of Pocobelle, the 300mw mail router

2009-10-05T21:24:00.000-07:00

Briefly: what Pocobelle is about - is trying to create the internet I thought (in 1990) we'd have in 1996. Back then, I thought the internet would be a network of peers - not clients and servers - but peers. What we call P2P networking is a bitter joke compared to what could have been, what could have existed on the edge of the network, in every household, in every business, in the USA and in Timbuktu.

I expected email, netnews, DNS, radio services, web services, etc - to all exist - inside your home and not out in the "cloud". I expected to be playing concerts via the jamophone with my neighbor down the street and games with a buddy across town - with sub 1 ms latency...

Yes, the world changed. Mostly everyone became cyberserfs.

I didn't. My dream didn't die, it was just resting. Netnews isn't dead, it's still there, lots of people use it. It's easy to run your own email server, less easy to run your own DNS... Radio... Well, one day...

It IS impossible to be a true peer on the internet with IPv4 without a lot of expense. I'm doing it (mostly successfully) on the cheap with ipv6. And... I'm trying to do it, with a 140 dollar arm box with 64MB of memory that's 100% solid state and eats 300mw. That's milliwatts. Less than 1/3 of a watt.

The machine's called Pocobelle.sjds.teklibre.org.

I'm not going to bother posting much about it on my blog here, if you want to see what's going on with Pocobelle, get ipv6 working, and see for yourself.

This is an excerpt, from a lot of writing in progress:

The Pocobelle project, day 10

Pocobelle has been acting as a backup email router for a few days now.

I get several hundred emails per day. It used to be thousands, but I switched to using netnews & gmane for my more high-traffic mailing lists, the lists that I mostly read and don't write to, such as lkml. Pocobelle successfully coped with the email I get in the dribs and drabs I get it in - I didn't have any complaints. It was transparent to me, and everybody. It did STARTTLS crypto without cracking 10% of cpu. My tests included sending a few dozen mails from a server outside my network, but that was it. Most of the time mail goes right to my laptop...

Today was the perfect day to try pocobelle out in a real world scenario.

I was without power or internet for 8 hours. My generator didn't work. (Most likely, I'm out of gas). My ice cream melted. Oh, well. I needed to defrost the chicken anyway.

The UPS that pocobelle was running on showed 95% of it's battery available after that period. I was really happy about that. Assuming I have a week when the internet stays up and power doesn't I should be able to have my email delivered without a problem, and periodically fire up the laptop to read it.

At 7PM, power and internet came back on simultaneously. I had previously turned off email to my laptop (and turned the laptop off, to save power). I booted up the laptop just to watch pocobelle do it's stuff...

Pocobelle got on the Net... Got an ipv6 address... And started getting the backlog of mail...

The Bind9 DNS server rapidly got to 23MB in size... The cpu went to 100%... 93% of it, gone, waiting for disk access.. the Loadavg lept past 5... available memory dropped to zero... My ssh session locked up...

Midway through the 15th email it bounced 3 messages, then it died.

Pocobelle ran completely out of memory and came to a screeching halt.

Sigh. The perils of engineering.

I hadn't thought deeply about the interaction between DNS services and email. Freshly booted, there is no DNS cache on the system.

I hadn't thought about a complete and utter cold start of absolutely everything pocobelle was connected to. There were no DNS caches anywhere it talked to that weren't "cold".

I'd got into a pathological situation, where the bandwidth being chewed up by all the mail being sent, and the time it took to "walk" DNS to verify it as "good" mail, competed and combined to bounce mails it couldn't do a reverse lookup on.

At the same time, the load on the system was such as to put it on the moon in short order.

As crashes go, it was not pretty. It brought back a flashback from 1995 where a CEO I knew, ecstatic with his new 26MB powerpoint presentation, emailed it to everyone in the company, and everyone he knew in the world, besides. That was an age, also, when a *good* mail server only had 64MB of ram....

1) Pocobelle only has 64MB of memory. (Pocobelle 2 will have 256MB or more) An easy "cure" for the memory problem was to enable swapping. When running without swap a Linux system will free up memory by discarding unused (read-only) program text pages, which are read-only, and swapping them in from the filesystem when they get used.

While there are a lot of binary pages you can do this to, it doesn't work on pages that have been modified by the linker, and it (especially) doesn't work on interpreted languages like python and perl. These languages often do have plenty of little-used pages, but they are *data* and can't get discarded because some day they MIGHT be modified further.

This arm build does not appear to have Jakub Jelinek's prelink utility installed, which will free up more memory by prelinking the various binaries. Prelink solved a few problems, but in the arm world, most people (I'm not) use a libc that wasn't compatable with prelink. I'm still researching this...

So, anyway, there are plenty of things that can't get swapped out that could, if swap was enabled. So I added 128MB of swap on the flash. Linux doesn't require that you have swap on a raw partition (although it is a good idea), so I just did a:

dd if=/dev/zero bs=1024k of=/etc/swap count=128
mkswap /etc/swap
swapon /etc/swap # and add to /etc/fstab

An even better cure for this would be to use a box with more memory but that's a problem reserved for pocobelle 2.

With swapping enabled, pocobelle grew decidedly less "chunky" in the general case. There is always a lot more free memory available for general use - for example, bind9 dropped from 23MB of ram down to 14MB. In normal use, 16MB is living on swap by default.

Whenever I get around to reformatting this USB key, I will put swap on a raw partition. I might put it on the built-in flash on the board, actually. We'll see.

2) Pocobelle was configured to use one DNS server - it's own - and forward to several local servers attached to my wireless network, provided by my provider. While this is a decent config... One that a normal client would use... given that all of the servers it was connecting to were ALSO freshly booted and ALSO had to walk DNS there, they ALL failed within the default DNS timeouts.

What I decided to do was establish a robust set of DNS servers (5), having pocobelle talk to itself twice - once in the beginning of the loop and another time at the end. In the middle it talks to my main mail server, which having already done the anti-spam protection in the first place should have a cached record of the remote server's origin already.

It should effectively put a 10 second timeout on the DNS lookup instead of a 2 second one, AND get to at least one server that has a good, primed, cache; a server in the US that's impervious to power failures.

(Getting this to work was a little tricky in that I'm using bind views internally to give me a consistent picture of my network and routing configuration(s), but I'm not going to go into that here)

I hope this is sufficiently robust. I'm not going to purposely instigate another 8 hour delay on my email, at least, not in the near future.

Another answer to this is to cache more of the internet's DNS service at the start, before accepting mail. (My mail is mostly not random, but comes from a limited number of mailing list servers). I have a buddy that used to cache the entire DNS root zones back in 1995. Maybe that's still possible.

It would be good to have some sort of cache log that I could replay on name service startup (or at certain times of the day, for example, shortly before I wake up in the morning) to prime the cache(s). I can sort of do this by replaying the mail logs through the DNS system, but it would be cleaner if I could figure out a way to get my top 100 sites out of bind periodically.

3) Given that write speeds to the flash are so slow it would be best to always keep at least 512K reserved for disk buffers. Smaller writes than 128k at a time are *bad* with flash.

I used to know how to do that, but the interface to the Linux swapper has changed so much that I have to google to figure it out. (Most of this blog was written after the power failed again)

4) Given that one of pocobelle's purposes is to be a mail router, and it lives on ipv6 which has little to no spam on it, it's somewhat pointless having even the minimal anti-spam services I have on it (like those reverse lookups that caused the bounces in the first place)

I'm not going to do that, I actually want to make this into a system capable of the best anti-spam measures I can come up with because spam is just never going to go away.

...

So, after fixing 1 and 2, I fired up rss2email on a new user on pocobelle. Rss2email is written in python. It took 12 seconds to start, and 16MB of ram, and was really going slow, so I decided that I didn't need to do that on pocobelle itself, but on my smart host elsewhere. Pocobelle just needs the mail itself, not the process that generates it.

Result: I got 25 messages as fast as they could be delivered.

It ran happily with 16MB of ram out on swap.

I'm happy with pocobelle today. I'm going to turn off my laptop tonight and see what happens.

AM Update: I turned the laptop on again, and got about 60 emails sent in rapid succession. The night before I'd double the default number of connections to 12 in a burst of optimism.... Pocobelle handled the load, but I think I'm going to limit the number of inbound and outbound connections to 4. At 12, it ran at 93% of cpu and got down to very little memory during it's burst of email. Pocobelle needs to remain responsive to DNS, in particular, as it's the main DNS server for the household, and has quite a few other things to do besides email.

Now, I'm running full starttls (encrypted) email inside of my household, which probably accounts for some of the cpu usage, but I think the overhead was of startup and running all those processes, not the crypto.

Maybe I'll try rate-limiting the number of inbound connections via iptables, tarpitting them maybe, to keep the mail server on the other side happy once pocobelle it gets past 3, keeping it from rescheduling the mail repeatedly. That will ensure a burst of email actually gets sent, albeit slowly. (this is also a good anti-spam measure)

On to figuring out 3 and 4...

ipv6 and smart(er) mail relaying in postfix

2009-10-03T10:01:00.000-07:00

A couple weeks back I started running most of the mail servers I am responsible for over ipv6. I posted a few notes to the postfix mailing list on that.

(My apologies for the excessively geeky contents of this blog recently, I have a few more "normal", real world. blog entries in the queue...)

I posted this question to the postfix mailing list today. (I got some good responses, more info the Updates sections)

I'm trying to wrap my head around a new problem - trying to have two postfix relays and a smart host co-exist where one of the relays is a tiny power sipping ARM based board... (Read on for details)

To recap, what I did was configure my in-house (and other servers I run) server to only listen and send on ipv6 via:

smtp_bind_address6 = my:ip:v6:ad:re::ss
smtp_bind_address = 127.0.0.1

And forward mail to my ipv6/ipv4 smarthost located in the co-lo facility via:

smtp_fallback_relay = [mysmarthost_onivp6.example.org]

For when that doesn't work. Postfix tries connecting directly to the given email addresses, which are usually ipv4, fails rapidly due to being bound to localhost only, then forwards to the smart host, for ipv4 hosts.

This handles the common case where people refuse mail delivered directly to them via ipv4 from invalid reverse dns, and hopefully works generically for those few sites (including my own) that exchange mail over ipv6.

That's been working pretty good. I'm not aware of having missed any mail at all since switching to this method. All the servers I control are exchanging email directly over ipv6 without the smarthost in the loop. I like it. Email is direct, secuire, and as fast as instant messaging once again.

Now I'm trying to wrap my head around a new problem.

Recently I built a 300mw (that's milliwatt!) postfix mail router out of an old 64MB ram TS7250 ARM board I had lying around and a 4GB usb stick, running debian lenny.

It works pretty good in my testing so far. STARTTLS Crypto works, it runs at the speed of my internet link (24KB/sec) without any problem, and transfers on the internal net at ~500KB/sec (it's bound by the usb stick, actually). I have not abused it heavily yet - I need to see what happens when I send very large emails, for example. I will have to limit the number of inbound and outbound connections, to be sure.

(I live way out in the country, and have a (slow) wireless connection to the net. Power and/or internet frequently go out. Remember the bad old days, when email got transfered via dial up connection or via carrier pigeon? Technologically, I'm living there, admittedly with a splendid view of the ocean.

Running a 300mw mail server makes a lot of sense - I have enough battery power to run for days instead of hours sipping power like that (the wireless router uses about 5w) It beats running mail on my laptop, at 65w, by a country kilometer.)

So what I think I want to do is setup fallback relaying as follows:

MX 5 mylaptop.example.org # if my laptop's up send mail there
MX 10 mytinyarmbox.example.org # if not, try my arm box
MX 20 mysmarthost.example.org # otherwise, default to my well connected host

Now, 99.9999% of the internet is NOT relaying mail over ipv6, so what happens in that case is my or your mail ends up at my smarthost, which then relays it for me.

Problem 1) I am under the impression from a foggy memory of reading some RFC or other, that at minimum, 2 MX records will be tried. So adding a third might introduce problems with some MTAs that ONLY do 2 MX records, in that far off day when more stuff speaks ipv6 directly, or when it
fails to fallback to my third, primary smarthost.

Update: Wietse Venema quoted me chapter and verse of the related RFC2821, which states:

"the SMTP client SHOULD try at least two addresses". With three MX hosts you're operating outside the recommendation.

More on how I currently solve this are going to be subject of another blog entry. Briefly, I implemented Bind9 "views" to present two MX records to the outside world, and 3 to my own. Postfix exceeds the RFC in every respect, and does the right thing. Solved. I'm routing my own damn mail.

Problem 2) My smarthost is only smart enough to try sending to one other relay (I think).

Problem 3) Similarly mytinyarmbox is only smart enough to try sending to one smarthost. I'm afraid if I set it up to relay it will fail to reach my laptop, then relay mail back to the main smarthost which will relay it back to the arm box which will relay it back to the smarthost until the loop count is exceeded. I guess I'm looking for some "never use the smarthost relay for these domains" option in postfix... Obviously, after googling, I'm not phrasing the question right....

Update: It turned out that the smarthost lines in postfix "do the right thing". It will not try to send email to a server that I control that has a lower MX record priority than itself. I couldn't find an answer on google, because smarthost does it right to begin with! Wietse, again:

If the machine sends mail to a less preferred MX host than itself,
then it is badly borked. To pull that off with Postfix you would
have to turn off DNS or override the routing with a transport map.

All I did was add another smarthost to my laptop, so when it can't get to my main server, it forwards the mail to myarmbox. In /etc/postfix/main.cf:

smtp_fallback_relay = [myreallysmarthost.example.org] [mytinyarmbox.example.org]

Bing! I can turn my laptop off 10 seconds after sending the last mail, and know that my mail will eventually get to the Net without my further intervention.

Now, it turns out I badly borked DNS the next day, and I'm still sorting through that, but that's not a postfix issue.

Problem 4) My laptop/primary mail server is actually on a dynamic ipv6 address (I control what ipv6 tunnel it is running on and update its dns record with nsupdate when it changes), so that no matter where I am, I have an ipv6 connection, when I have a connection. It seems inefficient
to route mail to my house and then back if I'm not there, especially when my house is off the net and I'm not there to fix it...

Update: The above problem is basically fixed by the dual smarthost line.

I am patently aware that there are other, less crazy ways to do all this (like fetchmail or offlineimap), but 1) I get a lot of mail (think: lkml) so getting email whenever possible, in the background, rather than via a cron job that eats my connection for minutes or hours at a time, is a good idea, and 2) I have to run my own mail servers anyway, so why not skip that step? And 3) It's kind of fun.)

If anyone would like to dink with this little arm box, email me privately, I'll set you up an account.

Squid 3.1 (with ipv6 support) lands in debian

2009-09-24T09:00:00.000-07:00

Last night I got an old box with an arm cpu mostly working. I used to use it as a dns and dhcp server back when wireless routers were lame. It's a TS7250 - a great little 200 Mhz ep9302 arm box, eating only 300mw of power (less with power saving on!), and now that I live deep in the boonies I figured I could retask it, maybe make it run email and squid, etc. Aside from floating point, it's probably about as powerful as a pentium II, and I used to hang dozens of email users off of one of those. I just need to support me and my roomate. 300mw sounds about right.

But it's gotta speak ipv6 to talk email in my world. So, first up was getting an ipv6 enabled and modernized 2.6.29 kernel... which I mostly have now, thanks to these patches for the ts-7250 which bring it up to date, enable 64MB of ram, and a host of other features on this board that I didn't even know existed.

Next was that all-important ipv6 enabled squid server. I've been building my own squid server with ipv6 support ever since the OLPC project started, and I'm kind of tired of it... and figuring out how to get it to cross compile for debian on the arm eabi I was NOT looking forward to.

This morning I downloaded the Squid 3.1 release and was preparing to get it built, looking over the bug reports in debian and in ubuntu and dreading having to build my own version for the arm box and...

I hit reload...

And Squid 3.1, with ipv6 support, landed in debian this morning. How cool is that?

Sometimes the net works in weird, wonderful, ways.

Building a virtual mesh potato, part 1

2009-09-11T12:32:00.000-07:00

Hardware for the mesh potato has been delayed another month or so (and I can expect another month for delivery), and I'd planned on starting work on it in September, so I was stuck. I have plenty of things to do but my mindset had turned toward hacking... I also wanted to hack my existing router into better shape for ipv6. The two tasks are congruent as they use basically the same OS and toolchain, so I spent most of the last two days building up an environment to work on it, without the hardware.

Enter qemu debian mips.

I spent a lot of time going down the wrong paths - for example it wasn't obvious to me that you needed an elf format kernel to run the thing. I also spent 24 hours of computer cycles trying to do debian install via the emulator, only to have the power fail 93% of the way through.

So, this morning, I downloaded a debian image for qemu mips. 24 minutes later I had a booting virtual mips box, emulating the MIPS to fool around in.

It Just Worked.

Embedded development just got a lot less hard. Flashing (and crashing) boards is a painful process easily avoided with an emulator. You can also do all sorts of robustness testing of the core software without fear of wrecking one of the precious few boards available, and you can work on "branches" of the core software by just keeping a couple copies of the emulator image around.

Heck, a good multi-core machine could probably emulate a dozen potatoes at the same time.

Once I get openwrt boostrapped onto it, it will let me fiddle with more modern kernels, alternate tools, and eventually the openwrt userspace that the mesh potato uses with well as the web interface, etc, long before the hardware is available.

I seriously doubt this is going to be fast enough to actually run the core voip software, but we'll see. The other core feature - mesh networking - may well be simulatable.

Maybe I can make a virtual box visible on the internet, too. That could help a lot of people.

My hat is off to the QEMU and Debian guys. You're awesome.

Back to hacking together a more modern kernel....

Accidentally cross building Linux 2.6.31

2009-09-09T18:06:00.000-07:00

I'm not really planning on re-entering kernel development. While intensely fascinating, it takes too long to compile on the hardware I have left to me, programming in it is finicky and painful, and getting fixes pushed upstream can be a nightmare, especially on the embedded architectures I used to do so much work on. It takes forever to download updates to whatever git tree I'm working out of, where I am, too.... I have a lot of trees and code lying around that let me build custom OSes quickly, but nothing worth publishing, all of it out of date...

...I am planning on kind of getting involved again, to whatever extent necessary, when the village telco ships some boards, but that's it. Well, that, and freeswitch. I keep telling myself that. I have to keep my involvement in charitable projects down enough so I can afford to eat, next year.

But I got a bee up my butt regarding dd-wrt. DD-WRT is a fabulous upgrade to many consumer grade routers, I've been running it (and its cousin, openwrt) off and on, since the projects' inception. They make consumer grade routers tolerable, with features like dynamic dns, local dns, sip routing, and ipv6. I installed the latest on my WRT300N a while back, and it's quite awesome, a real step up in sane additions to your home network.

I've also been fooling around with ipv6 a lot ever since all of google got on it a few months back, and dd-wrt almost - kind of - works - well enough to tolerate. The default dd-wrt doesn't contain ping6 or iptables6 or a current version of radvd and the web server doesn't do ipv6, and there's no dhcp6 server, and I'd really like to get bind running on it... and... it bugs me and...

... the dd-wrt kernel on this particular router is still at 2.4.X, which is so obsolescent it hardly bears thinking of. Worse, in dd-wrt - that kernel gets compiled with an ancient version of gcc - 3.3 - and there was just no way to get the 20GB+ svn checkout of dd-wrt from there to here without a lot of pain I still can't bear thinking of.

Fiddling with toolchain incompatibilities is something that I'm not even willing to get paid for nowadays. Fixing code to compile is not coding.

There has been a lot of changes to IPv6 that landed in the 2.6 kernel and didn't get backported to 2.4, too. In particular, so far as I know routing announcements got made more sane (so default routes should "just work"), mobile ipv6 (which kind of appears to be a dead project) got incorporated, and all this other cool stuff is in there too (and yea, I'm the type to get excited about fannotify).

I had a little spare time this afternoon...

So, on my well connected host (thank YOU isc!), I pulled down Linus's entire git tree today. It took about 4 minutes. kernel.org is on a GigE network 4 hops away.


traceroute to git.kernel.org (204.152.191.40), 30 hops max, 60 byte packets
 1  guest.r1.sql1.isc.org (149.20.54.2)  1.469 ms  1.842 ms  1.832 ms
 2  int-0-1-0-0-606.r1.sfo2.isc.org (149.20.65.3)  6.497 ms  6.522 ms  6.502 ms
 3  int-3-0-0.r1.pao1.isc.org (149.20.65.129)  4.059 ms  4.070 ms  4.050 ms
 4  git2.kernel.org (204.152.191.40)  1.757 ms  1.744 ms  1.725 ms

It would probably take a few days to pull the tree where I live now, and forever where I am moving next week. I try to tradeo-ff bandwidth issues with rsync a lot, pulling down source trees to this machine, then (slowly) pulling them down to my home machines. That way I can handle partial transfers and eventually get what I need to home base, but multi-GB downloads are no fun, regardless...

I configured a basic kernel for the mips + broadcom architecture that's in my WRT300N, did an apt-get install of emdebian's cross development infrastructure (another 20 minutes because it had to come from France) and the gcc 4.3.2 cross toolchains for mips and arm...

and fired off a batch process to compile the kernel.

batch <<AAA
cd ~/src/git/linux-2.6; make O=/home/d/build_wrt CROSS_COMPILE=mipsel-linux-gnu- vmlinux modules
AAA

I may be horribly retro for using batch but I love it - I can queue up lots of work for (this very slow) processor, go offline, and get notified via email when each job finishes. (I configure atd to default to a high load average so it handles the multiple cores better) I queued up a download of the latest dd-wrt tree, too...

About an hour and half later, my email dinged. There wasn't a single error in the compile. It generated the modules, symtabs, and vmlinux, apparently perfectly. Amazing. Compiling for alternate architectures never looked so clean before.

It can't possibly be this easy. No, I don't want to take this further. There's probably definitely something that just won't work on 2.6 on this router, some proprietary module somewhere, and that's why nobody's done it yet... No, I don't want to try booting this kernel on my WRT. No... somebody stop me... please...

I had entered the kernel development schedule into my org-mode so I can surface every so often and check in on things, and I was kind of aware that Linus was planning on releasing 2.6.31 "soon". It turned out that about 5 minutes after that build completed, he released the that exact same code with the change to the makefile calling it 2.6.31...

Now I have to recompile with the right version number, so I did a make clean, and fired off the same batch process.

And checked in on my svn checkout of the rest of dd-wrt. It had hung partway through... but enough came down for me to figure I really could try a new toolchain through the whole thing, update busybox, radvd, and maybe take a stab at cross compiling a minimal version of bind...

Do I really want to rebuild my entire router from source code and move ipv6 another iota towards acceptance?

I just checked the surf report. There's no waves til saturday. I'm doomed. I Sure wish I had two of these wrts available so I could brick one - or a working qemu emulator. Yea, lacking those, it would be insane for me to continue... I *NEED* this router just to exist on the internet...

batch <<AAA
cd ~/src/qemu-0.11.0-rc2; ./configure; make -j 2
AAA

I'm going to dinner now. I need another distraction, fast.

Update: I got back from dinner and went right back into deep hack mode until 6AM. I managed to build an ipv6 enabled version of openwrt for the mesh potato, but have totally failed to get a working mips emulator going...

I guess freeswitch is next. And maybe some more sleep.

Beach calm #1

2009-08-26T20:02:00.000-07:00

The day I published my last blog, I'd made my top priority finding a new house to live in.

I got up early to go to La Playa Yanqui. There, I met Alan, the proprietor of a B&B there, and we had a lovely discussion about just about everything... just him, me, and his dog, on that beach, by ourselves, shortly after dawn.

He mentioned that his neighbor's house was up for rent in September.

I got a nice surfing session in. The water was clear and warm. The waves were small and otherwise empty of other people. There were a few jellyfish, but they didn't sting that bad.

I got back to the car and broke the key off in the lock. Disaster, right? You're at an empty beach, all alone, on the wrong side of a mountain, and your cell phone is on the other side of a glass window and wouldn't work anyway, even if you could get at it. You're tired and hungry, in the middle of nowhere, you've spent twenty minutes with exactly one person who lives somewhere nearby but you don't know where that is and you have places to be and things to do and your key is broken and there is nobody around and you have to get out of there somehow as soon as you can so you can go deal with life itself...

I tried a variety of ways to get into the truck, and failed at all of them.

I sat down to calm down, and then walked over to some rocks, dangled my feet in the water, looked at Costa Rica, only 18 or so miles away... and I realized that I really didn't have to be anywhere, or do anything at all. Somebody would come along... or I'd walk over the mountain... and it was a damn beautiful day. The sun was shining, there was that early morning stillness, birdsong, and waves lapping gently on the beach... and I was getting some energy back...

And, after a while, I realized that I really wouldn't mind at all if I had a string of days with nothing to do and nowhere to be, just like this one.

I was more trapped by my need to be elsewhere than by where I was.

The answer to getting the door unlocked finally came to me, and in a few short minutes I was "free" to go home and do whatever it was I thought was so important I get at earlier. I couldn't for the life of me remember what it was.

Just as I finished packing up the car, another solitary surfer showed up.

I wished him the best of all possible days.

...

On my way home, I went up and visited Alan and his wife at their lovely, calm, wonderful B&B that they have spent 9 years building with its fabulous view of the ocean and Costa Rica. (I'm totally embarrassed to have forgotten the name of it, and it is completely off the intertubes)

They offered to show me their neighbor's house, but I deferred - I had places to go, things to do - and I couldn't let myself hope, too much - maybe it was a shack with holes in the roof, deep in the forest - I mean, that was sort of what I wanted, something that Henry Miller would have dug, but I kind of needed lots of electric power....

But by 11AM. I'd maybe solved the biggest problem I had, a problem I'd had for over 10 months, without even trying, totally by accident. It was magical.

When I got back to the chaos and confusion of life in town... my main computer - all 900 watts of quad core and and triple screen technological powerhouse, had died.

I can't honestly remember when it died, actually, it was kind of traumatic and I wasn't coping with it well. Maybe it heard I was thinking of going off the grid and euthanized itself? I'd gotten very dependent on having three screens available for my work, in portrait mode. That power-sucking howling technological mass of terabytes was always on, full of things I needed to be working on... but I didn't need to be working on them that day. I got on my decrepit laptop instead.. and had dozens of birthday wishes in my mailbox. It was awesome... I got too many wishes to reply to them all and was done with the computer long before 1...

Later on I played a whole lot of music - and had a lovely lunch and dinner with some friends.

I can't remember what else happened that day, but it was a perfect b-day.

The death of my main machine actually opened up a whole new realm of possibilities for me, but I'll get to that in my next blog.

(my apologies to Alan, I will rewrite this piece with the name of his B&B as soon as I remember it)

Designing an architecture for 2010 and beyond, for me

2009-08-11T01:00:00.000-07:00

Here I am. A Birthday Boy again. 44 years old. Ancient. Ugh. Unless you take the speculations of the Exotropians and Singularitists seriously, I'm well past the midpoint of my lifespan. I maybe have 20 years left to me. I mostly feel like I've lived a lifetime already. I used to have to work 18 months a year to get by. I used to take down a lot of money, and yet was still broke all the time. Now... I live in the third world and work on charitable stuff a lot and am still broke all the time. I daydream of making 3k a month for the most minimum effort I can make while sticking to my principles. I have like 20 plans to do so, most in various states of disrepair, hung up on one thing or another.

I tell myself, that for the past 5 years, I've been working on straightening all that out.Myself tells me - "Get with the program!" and "What have you done about all that, lately?"And I reply:The school next door starts up at 7AM, 6 days a week. 200 niños y niñas are on a permanent screaming recreation break, it seems, from start of school till 5PM, and then the marching band practices from 5 to 8, and after that, SJDS is usually partying down, loudly.

Larry Niven theorized with his Protector alien, that at mid-life, protectors lacking family would have to adopt their entire species to have a purpose in life, and those that didn't adopt their entire species, died young.

I could use a little less of my species running around screaming.That doesn't count the fan that's always running, either. What little work I've been getting done has been done nocturnally as a default and I'm often sleep deprived.

Moving to the house I'm in has to rank in the top 10 of the worst mistakes I've ever made. Every conscious minute I have had for the past 9 months has been filled with external noise, except when I was surfing. "Excuses!" - myself tells me.The peace I get from surfing doesn't last more than a hour or two once back in town, and it's not like I'm getting any work done out on the waves. (well, my belly has mostly moved to my chest) I can't surf every day, as much as I would like to, because there isn't surf every day, nor is my body up to it. Every other day is feasible, every third day, probable.

I got the opportunity this week to house-sit at a house that's way off the grid, deep in the country, overlooking a valley, near an awesome surf beach, without a single electric light in eyesight at night.

I wrote most of these past two blog entries by candlelight, in Emacs's org-mode. The crickets and howler monkeys are the loudest thing I can hear. It's wonderful. Moving back this far into the country has problems - I'd need a 4WD car and more capital overall - but perhaps just moving to a quieter neighborhood in SJDS will suffice - I do like not having to manage a car - but even those solutions begs the question: Jeeze, what do I want to do with the rest of my life?

Let's review.

The year so far... Professionally

The year so far, despite my disgruntlement on this day, has been a good one. I worked on a pretty cool music application that I can't talk about publicly. I may do a bit more work on it later this year.

I got stuck on it, though. I handed off the work to a programmer I deeply respect, and although he made serious progress, he didn't quite make it work, either. I feel perversely good about that.

After years of trying, two of the Linux drivers I wrote finally made it into the main kernel distribution. Not a lot of people can point to their name in the kernel. The ardour interface to one of them works well in ardour for the few users I have, but need a bit more work to make it into ardour's mainline. I'm told people are working on rosegarden and mplayer support as well. I'm happy. I wish the darn drivers were done so I wouldn't have to think about them anymore but no-one has shown up to take this cup from me yet. I'd like to finish up the first driver entirely and start on the ardour interface to the second one.

Visits

Much to my surprise my cousins, Laura and Linda - who are the most adventurous members of my family, by far - came to visit me in Nicaragua. They had a great time.

My gf from HS (She Who Will Not be Blogged) also visited for a week. This was mind-blowing, she's a New Yorker, New Yawkers just don't come here... She had a hard time with the heat, but had a good time in the night times. I hope I can lure more friends and family to visit in the coming years. The place really does have many charms, charms that will be paved over in 20 years. Some of my family even surf (yes, I'm talking to you, Bill), and others dive (Steve!).

I've got a little LISP, I've got a little LISP

For the past three years I've been writing down a long survey of what computer languages I liked and disliked, what was right about them, and what was wrong, what I did with them, & why I stopped. I got caught up on the bleeding edge of functional languages with experiments in gocaml and scalia, and also explored some unusual processors, like the Blackfin and Propellor, all in the hope that I'd find a language that tied to an obvious future generation of processors that would scale *me* into the next decade.

No such luck.That document not ready to be published, not by a long shot, but the list of languages I worked in is rather impressive, and the ones I merely played with even more so. All of which were useful and interesting inside of their domains and fail at one of the requirements above.Lately I've been working mostly in Spanish, which has been harder for me than all the computer languages I've worked in, combined. You don't have to pronounce "C", and all the weird grammar rules in Perl pale beside Spanish (and, now that I look back on it, English), and I still speak Spanish in a terrible pidgin that suffices for discussion of politics, women, music, food, and little else. After programming for nearly 25 years, it's second nature to view the answer to my problems is more programming. That didn't work. I came to the reverse conclusion a few years back that the answer was LESS programming, preferably, none. That didn't work, either. Now, I'm trying to find a happy medium. I wrote about discovering org-mode recently. It's helping a lot for re-programming myself.What I want to find is a language that has the equivalent of "set bugs off; do what I'm thinking", something simultaneously fast to write in and fast running, easy, scalable, and reliable. The age old answer to that is you can meet any two of those requirements with any language, easily, the third, with a bit of work, and the fourth and fifth, not at all. Pick any two to start with.
Most of what I worked on and in for the past decade executed fast and was not easy, down-right hard, in fact. It paid well, but the price was in language expressiveness and in my personal over-specialization. Once I was a gtk 1.x expert at a time when there was only three paid jobs in the entire field! I jumped on the voip asterisk bandwagon early, which paid for a while, and has now mostly died in the wake of Skype (but fear not, SIP continues to progress!).Right now I'm overspecialized in performance optimization (which nobody cares about) and real time programming (which few care about), and Embedded programming, which is all the rage, is mostly getting outsourced, for some reason. The iphone is hot (and I'm kind of sorry I missed out on that), as are cell phones in general (but I was involved in that back when it was hard, in 2000, and don't even own a high end cell phone now) I can't bring myself to care about cell phones. Webpads maybe... I have a story to tell about that, one day... I was still deep in Perl when the trend went to Java (I'm not sorry I missed out on java), Before I decided to take this summer off, I was working on making libsamplerate "do" SSE. SSE programming is hard - it is the equivalent of writing assembly in C without the benefit of a macro assembler. Hard. My early tests showed significant improvements that I think will bear out after I get 3 days of full brain cells to actually make it all work, and a couple weeks more of less brain cells to clean it up - but to do that I have to reinvent somewhat more C++ inside of C than I'd like. After it is *done* I'd like to not write in assembly any more. Maybe the compilers will catch up to SSE more in the future.

Hell, I'd like to ditch "C". I have multiple projects left to finish in C, however, and I don't see how I can quit using it - but I do hope I can use it a lot less in the future. I was tempted a lot by what's going on in Cuda and OpenCL- massively parallel programming is going to be the answer to some major problems - but after trying to wrap my head around doing libsamplerate in Cuda - I want to move up a level. (or quit the field entirely - how many industries are you aware of that require you re-invent yourself every five years?). C++, well, ardour's written in it, I still have bugs left to fix in ardour, but I'd like to quit that, too. Ardour rocks but it is extraordinarily complex under the covers.

One major program left to write

I have one fairly large program of my own, left to write, one that I must write, in some language or combination of languages I'll decide one one day soon. I keep hoping someone else will write it, but so far, no luck. I'm not going to talk about it today.
The biggest program I have left to write, though, is the one my life runs on, and it's barely booting right now, and pretty buggy. Some percentage of my output has GOT to get paid for. What that is - whether it be the music recordings I've done, the coding I do, the writing I do - or something else entirely - remains to be determined. What have I got that other people want?

Conclusion

In less than 10 hours spent on these two blog posts, I pumped out 5000 words, clocked in and out over three segments of time. I like knowing that. 500 words an hour, if it were paid, is a living at 6 cents a word. 10 cents would be better, of course, and some markets pay that.Doc's "Because" effect isn't working for me because I just don't generate enough output to take advantage of it. I need to follow Heinlein's dictum - "Sell what you write, write what you sell".

Itunes pays 33 cents a song, roughly, which has to be divvied up between the label, and the artist. Other services pay more. Both Apple and the major labels are trying to resurrect the album - which is great - I LIKE albums, I have recorded 9 albums of various Nicaraguan artists - structured as albums, not singles - in the queue now, and have been experimenting with multi-mode CDs - a normal CD with an additional filesystem on it containing pre-made mp3s, graphics, and hyperlinks. Coming up with the dough to get even one of them manufactured is a problem.

I'm going to take the rest of this month to work on me - via automating every task I can think of, mostly using Emacs LISP, taking the time to solve the hard problems well, for a change, and documenting and backing up the results so I will have them forever. It is a surprising conclusion, that the language I'd (re)learn this year would be any variant of LISP. It's remarkably retro, in fact, and not something I'd expect others to do. But I think better in LISP, and I need solutions that apply to me. I suspect I'll explore some more current variants of LISP - arc and Clojure come to mind... Clojure has a lot of appeal because of it's interface to the java virtual machine....And hopefully, in September, I'll find some paid project to work on. I've got 3 candidates. I've already lined up enough charitable work - notably the village telco is sending me a couple of their cool boards shortly - to keep me busy for years and years.

I don't how I extract myself from my current lease, I have to get into a new house as soon as possible. I have a shot at working out of Italy, perhaps, and I'd like that a lot, too, so I'll work on solving the housing problem via one method or another before my lease is up.The year is still young, even if I am not! I'm going surfing at dawn.

Update: It's dawn. I'm outta here.
Update (Aug 16): I tried to edit this piece with the blogger API. It did bad things. I will fix it later. Sorry for the loss of most paragraphs.
Update 6 Hours later: Fixed now.

Going retro, re-adopting Emacs!

2009-08-10T18:53:00.000-07:00

After the web hit me over the head in 1991, I started adopting one new shiny tool after another. My desktop was filled with a zillion browser windows (since replaced by a zillion browser tabs), a zillion chat clients (since replaced by a multi-chat client (pidgin)), skype and ekiga, Thunderbird (for email), an RSS client, ardour, a mp3 player (presently aqualung) and... Emacs for programming and writing. My bread and butter came from Emacs. My wine and cheese came from Emacs. Everything else was just shiny.

I'm writing this blog today, in Emacs, in glorious plain, green text, with a black background, without a single graphical gutter, handle, button, gee-gaw, etc, in sight.

Green on black is a cool, calming set of colors that lets me deliberate on things - (even if some of the best rants ever came from a green screen in Jwz's Gruntle)

I have - oops, I need to take a call - ker-chunk - context switch - and "oh! I have mail!" - ker-chunk - and - ker-chunk - someone on irc needs attention - ker-chunk - check out that new app! - and ...

... and before I know it, the day is shot. What Paul Graham wrote about the Maker's Dilemma really applies to me.

When you're operating on the maker's schedule, meetings are a disaster. A single meeting can blow a whole afternoon, by breaking it into two pieces each too small to do anything hard in. Plus you have to remember to go to the meeting. That's no problem for someone on the manager's schedule. There's always something coming on the next hour; the only question is what. But when someone on the maker's schedule has a meeting, they have to think about it.

I also need to relegate activities into work and play modes, and current window systems and tools are emphatically not designed to do that. They are there to dominate your attention. All the applications and web pages nowadays come with shiny, bright-white backgrounds. This is an exciting color! All the shiny apps call to you, saying "use me! use me! use me now!!!!" when I have better things to do. They all want the whole screen to play in, too. "Use me! Use Me! Use only me!"

I really want to dominate my applications and schedule my mind-slots for them in the times when I have brain cells, not vice versa.

I have ADD, bad, in the first place, and an attention deficit on paying attention to my attention deficit going on five years ancient. A five year old has a bigger attention span than I do.

I keep trying to cut the applications I use down to a bare minimum so I can switch between them with a minimum of context switch. Lately chat, rss, and email has been dominating my attention. Even 5 applications is too many - there's also the application I'm working on, whatever it is, to compete with for screen area.

Last month, I decided to try and make Emacs do more of what I wanted. I'd try to get rid of everything else - chat, mail, rss, and web...

Along the way, I was ompletely blindsided by the capabilities of a killer app for Emacs - Org Mode.

I have been whining about my need for a decent outliner for nearly a decade now. (I did not just whine, I also tried to write one in gtk) Word and OpenOffice seem to think that a screen slide's worth of outline is enough - which it isn't for bigger ideas, and all the other modern day outliners I tried were slow and kind of chunky. They didn't work how I wanted them to, in the environments I need them to work in.

I concluded that the kind of outliner I wanted, typeless, with all sorts of weird relationships between the data, was best written in assembly language, and my best shot would be to try and port something like MORE...

And I really didn't want to do that.

It turned out I'd had my perfect outliner installed all along! org-mode! I wrote this (and all of the upcoming blog entries) in org-mode, I'll get back to raving about org-mode in a bit.

Killing chat

I got off of the pidgin multi-client chat pretty easily. I use erc inside of emacs (irc is my primary chat system anyway). Adding support for msn, jabber, and yahoo I did via bitlbee, which is a universal server gateway for irc.

It turns out that I've forgotten most of my chat client passwords, and I haven't figured out how to get them back. Score!

The only thing missing from the default install of bitlbee was support for the Off The Record messaging extension, which thoroughly encrypts the chat conversation using source code I have a reasonable degree of trust in.

I like using OTR, a lot. OTR makes me feel much safer than almost any other system. Consider how Skype operates in china and think upon how much MSN, yahoo, etc must know about you... and how similar your government has become to China... and install OTR - it works with trillian, pidgin, and a few other chat clients. Bitlbee can be used if your client doesn't support it directly. You'll talk freer, trust me.

Getting that to work in bitlbee on jabber and yahoo required a custom compile of bitlbee, and has trouble with html formatted messages, but that was not much of a problem compared to the added security.

Along the way I added a keystroke (control-scroll_lock) to send stuff directly to the pastebin. Pastebins are really useful collaborative tools.

I have two chat systems left that I intend to minimize my usages of.

- The in-browser chat that facebook uses. It's buggy and slow and I hate it. I'm giving facebook the boot from my life, just like Valerie Aurora just did. To all my facebook friends: Sorry. Contact me via irc, msn, yahoo, or jabber if you need to chat. I'll get mail notifications, too, but please switch to sending email from a real provider. My email address is published on facebook. Please use it.

- Skype. I can't see how I can get away from skype, I'm hopelessly dependent on it, so it stays. I have, however, taken advantage of it's scripting capability to be able to log calls from skype into emacs to some extent. Next up, is to be able to make calls from emacs and more importantly, get it to co-exist with my other audio applications.

Killing the Bird

I'm almost completely off of Thunderbird, and am now using Emac's GNUS for sending and receiving email.

Prior to using Thunderbird (and mozilla), I'd used Eudora, and in-between I had a flirtation with gmail (OK, OK, it was a committed relationship) and PHB-like things such as Evolution.

Both of my primary email programs had a single feature that I was addicted to: Mail Filters. I have 3 dozen mailboxes that I use just to handle the volume of incoming mail I get, and something like 7 accounts. It's a real problem, even though I've got off of most of my mailing lists over the past couple years. I always found the procmail language error prone and difficult to use, so I discovered that a newer and simpler language, sieve, was now the default on ubuntu 9.04. I got the thunderbird filter exporter working, and spent an hour translating all those filters into sieve, and another hour debugging that...

... And about 10 hours trying to make it work right and work with Maildir and mbox format message boxes locally and over imaps, which didn't work, nor did it work on my older (8.10) ubuntu box, but I have high hopes to make it work, later. Maybe I'll do a custom compile of dovecot, or wait for another release. Right now I'm running without filters and I hate it. I'm almost ready to write some rules in procmail.

Emac's GNUs mail reader has seven features that I like - it doesn't show you read mailboxes by default, it (optionally) expires mail, it integrates with netnews and rss, I can get to it with a keystroke from emacs from anywhere, it does crypto, and it's very fast. The seventh feature I like is... it's very damn fast. Now that I have local email being spooled and received via postfix over ipv6 (no imap!) it's amazingly fast, as fast as instant messaging, and with leafnode and the local mail store... I process mail, news, and rss off the internet by default!

It required a lot of work to get this far, and I'm still not quite done, so I'll write more about that later.

Think about it, though, yourself. Why does sending email have to be slower than IM?

Enter... org-mode

.
Exactly one month ago I discovered that emacs had a very active project for a super-outliner... called org-mode. I fired up my emacs-snapshot, hit M-x-org-mode, read the manual, and fell in love.

Actually, I didn't read the manual. I sat down and just started typing and all the basic keystrokes came naturally. I was days along before I needed the manual. Later on, I'd think: "There must be some way to hoist a region" or "Schedule a repeating task", and there the intuitive command would be, in the manual.

I'd already been building the latest releases of emacs in anticipation of the 23.1 release... (Emacs's release cycle has slowly been getting out of a slow crawl into more of a stroll, which is suitable for a middle-aged program)

I just finished an outline of my entire life, of every project I have, of every task that I need to be doing, that is nearly 2000 subheadings along, 10s of thousands of words pulled from various sources. It will probably be 4x that long before I'm done, and that's before I add blogging capabilities to it.

It's full of to-do lists, ideas, and checklists, and needed items and budgeting. I hit two keys (F12 a) and I can see the week at a glance, or two keys (F12 t) - to see all my short todos that I could do when I have spare time, clock my time spent on any of it with another two keys, and stay on task with another (single key). Best of all it's all plain text, under git source control, so I can handle my words and tasks just like source code, and periodically checkpoint my status and push it out to a backup server, which I'm doing via cron. Heaven. For the first time in ages I'm not dependent on figuring out how to backup and track a zillion different types of files and databases in a zillion different applications.

One extremely fun thing about org-mode is that the books I've long been writing have timelines associated with them. After I get done importing it into org-mode, I can set my agenda to the far future (it starts in 2012 but most of the action is in 2022-2080) and see what is going on that day.

I've also plunked the arrival dates of the Dawn Probe into it. I can see my way clear to automating the light-seconds of several asteroids relative to each other against the date tags, so I can fiddle with an automated plot against the orbital parameters.

Remember mode and templates

An old friend, rediscovered, was Remember mode, with template support. Does anybody but me still remember how useful Sidekick was? Emacs's Remember mode is WAY more useful than Sidekick.

I can hit two keys (C-c r) to interrupt what I'm doing with a stray thought, and file it away somewhere else so I can deal with it at an appropriate time. Two more keys (the ubiquitous - "do the right thing" command, C-c C-c) and I'm back inside of what I'm doing without touching the mouse.

It's still not quite perfect. I'd like to do costing and budgeting - so I could schedule purposes of an item for when I need it, when I have the dough. I'd like there to be a better default scheduling estimate associated with several types of todo entries (which org-mode supports, I just haven't figured out the right template and timestamp sexp), and the start and stop times don't quite work the way I want them to, yet.

The custom templates code currently looks like this:


(setq org-remember-templates
      '(
 ("Bug" ?B "* BUG %?\n  %i\n  %a" "~/org/BUGS.org" "Bugs" (emacs-lisp-mode))
 ("Journal" ?j "* %U %?\n\n  %i\n  %a" "~/org/JOURNAL.org" "X" my-check)
 ("Idea" ?i "* %^{Title}\n  %i\n  %a" "~/org/JOURNAL.org" "New Ideas")
 ("Web" ?w "* %u %c \n\n%K\n%i %!" "~/org/bookmarks.org" "The Web")
 ("Blog" ?b "* %u %c \n\n%K\n%i %!\n :PROPERTIES:\n :Effort:\t0:10\n:END:" 
 "~/org/blog/blog.org" "The Edge")
 ("Private" ?p "* TODO %^{Description} \n :PROPERTIES:\n :Created: %U\n :Backlink: ->%a\n :END:\n\n %?" 
 "~/.Private/Private.org" "New") 
 ("Todo" ?t "* TODO %c%?\n SCHEDULED: %u\n\t:PROPERTIES:\n\t:Effort:\t1:00\n:END:" 
 "~/org/Do.org" "Do")
 ("Quick Todo" ?T "* TODO %c%?\n SCHEDULED: %u\n :PROPERTIES:\n :Effort:\t0:10\n:END:" 
  "~/org/Do.org" "Quick Do")
 ("Handheld" ?N "* TODO %c%?\n SCHEDULED: %u\n :PROPERTIES:\n :Effort:\t0:10\n:END:" 
  "~/org/handheld.org" "Stuff")
 ("Quick House" ?H "* TODO %c%?\n SCHEDULED: %u\n :PROPERTIES:\n :Effort:\t0:10\n:END:" 
  "~/org/Do.org" "Quick House")
 ("House" ?h "* TODO %c%?\n SCHEDULED: %u\n :PROPERTIES:\n :Effort:\t1:00\n:END:" 
  "~/org/Do.org" "House")
 ("Asteroids" ?a "* %u\n %c %? \n%:region" "~/org/bookmarks.org" "Asteroids")
 ("Ardour" ?A "* %u\n %c %? \n%:region" "~/org/Ardour.org" "Remember")
 ("Space" ?s "* %u %c %? :SPACE:\n\n%:region" "~/org/bookmarks.org" "Space")
 ("Emacs" ?e "* TODO %c %? :EMACS:\n SCHEDULED: %u\n :PROPERTIES:\n :Effort:\t0:10\n:END:" 
 "~/org/emacs.org" "Remember")
 ("FIXME" ?f "* TODO %c %? :FIXME:\n SCHEDULED: %u" "~/org/Do.org" "FIXME")
 ("TEST" ?Z "* %c %? :TEST:\n SCHEDULED: %(org-time-stamp +1)" "~/org/emacs.org" "Remember")
 ("Phone" ?p "* DONE %? :CALL:\n- STARTED: %T\n STOPPED: %T" "~/org/calls.org" "Calls") 
 ("Get" ?g "* TODO Buy %^{ITEM}\n%^{DESCRIPTION}\n- DEADLINE: %T %?\n :PROPERTIES:\n :Effort:\t0:10\n :COST: %^{Cost}\n :END:" "~/org/buy.org" "Buy") 
 ("Chat" ?c "* DONE %? :IRC:\n- STARTED: %t\n STOPPED: %t" "~/org/calls.org" "Chat")
 ("Delegate" ?d "* TODO %? " "~/org/delegate.org" "Delegate") 
 ("Gnugol" ?G "* [[gnugol:%^{Keywords}]]" "~/org/gnugol.org" "Search Later") 
 ("Procedure" ?P "* %^{Procedure}" "~/org/Procedures.org" "Procedures") 
 ("Nicaragua" ?n "* %^{Heading}\n%^{Text}\n %c %?\n %u" "~/org/book/nica.org" "Notes") 
 ("Ipv6" ?6 "* %^{Heading}\n%^{Text}\n %c %?\n %u" "~/org/ipv6.org" "Notes") 
 ("Vocab"   ?v "** [[dict:%^{Word?}]]\n%?\n"  "~/org/Vocab.org" "Vocabulary")
 ("Someday"   ?s "** %^{Someday Heading} %U\n%?\n"  "~/org/JOURNAL.org" "Someday")
 ("Appts" ?a "* Appointment: %?\n%^T\n%i\n  %a" "~/JOURNAL.org" "Appt")
 ("RT"    ?r "* [[RT:%^{Number}][%^{Number}/%^{Description}]]" "~/org/rt.org" "RT")))

Adopting org-mode has put a lot more structure on my life than I had before, and now ideas, tasks, todos, thoughts, blog entries, etc are all going into more appropriate bins where (someday) they may get acted upon.

Another favorite feature (I have plenty more to write about) is that I can get stuff into org-mode from the command line or a web browser. One swipe of the mouse, one click on a special bookmark in firefox, and I've slammed a piece of useful information into the right spot in my personal database.

I know emacs's keyboard command set would offput anyone that was used to pointing and clicking for everything, say, most of those under 35, but for me, emacs + org-mode is heaven, found. Org-mode's default keyboard assignments are a natural for anyone that has ever used a good outliner, you hardly notice you are in Emacs...

Proof-reading and text to speech

Although I was a persistent reader of netnews, I never posted that much. I usually write several drafts before I'm satisified with what I wrote. Usually that takes days. I almost never post something late at night - and always regret it when I do.

Netnews has a feature that lets you supersede a message in transit but at the time I used netnews a lot I don't think it was available.

I'm grateful for blogging in that it helped me get over that first draft fear, and mad at it for not handling the follow on work well.

I'd like to take my existing canon and re-organise it, now.

I have always done better with a proof-reader. I have a terrible tendency to miss articles (the, a, etc), and mangle tenses. These problems become obvious when read aloud, but it is kind of tedious and still error prone for *me* to do the work.

I have used various speech synthesizers (festival, flite) over the years, as well as speech recognition software. In the latter case, well, if Dragon worked better under linux I'd probably still use it, but I tired of wearing a headset all the time long ago.

I settle now for dictating into a recorder and (if I have the money), sending it out to be transcribed via Transcribr. I've got hundreds of hours of me yammering into a recorder in the queue. Worse, they are mostly uncategorized, not even dated, and in wma format, which is hard to deal with. I need to come up with a method of sorting though it all and categorizing it. Getting to where I have one-key delegation to transcribr would be good, too.

In the former case - speech synthesis - most of the open source solutions don't sound very good - and had a hard time interfacing with out sound software that I was using, and almost never with the editor.

Back when I was hacking on asterisk, however, I discovered cepstral - which is founded by the same guys that released festival as open source - and that they'd found a business model of releasing domain specific voices that are quite good. I bought two ("Amy" and "Diane") and installed them. I think Diane sounds the best (but am tempted to try a British voice next)

I found something simpler than emacsspeak - speechd.el - and got it running. It's still too complex for someone that is still sight-enabled, so I simplified matters with little custom lisp,
(myrtb () "Read That Back" ...)

Now, in emacs, I mark a region and hit a function key - and I get everything read back! I can step away from the keyboard and listen to what I wrote while doing something else, like making coffee.

Really successful writers always point to their "first reader" as being key to their success. Usually that person is a wife (or husband) or close friend.

Lacking that first reader, my first reader is speechd + cepstral.

I also have some cute crontab scripts now like "say-nag" which nags me every 20 minutes with whatever I'm supposed to be doing in ~/.nag, and appointment notifications are spoken rather than put on the screen.

...

I have a lot more to write about org-mode, etc, but I need to switch to some new blogging software and disqis next, so I can truly do push-button publishing once again.

Without a browser.

This is for the sysadmins

2009-08-01T14:30:00.000-07:00

A little late for Sysadmin's day...

But the radio play of Cory Doctorow's "When Sysadmins ruled the earth" had a reading of the Declaration of Independence of Cyberspace that brought a tear to my eye...

and made me play Redemption song and reminisce fondly about alt.sysadmin.recovery.

Then, all suffused with all that nostalgia, I went back to re-read Barlow's canon, via the wikipedia link.

Somebody will login to wikipedia and fix that link, I'm sure. If someone else doesn't fix it, I will, but not today. A lot of people care about the Net, maybe not in the same way we did back in the 80s and 90s, but they care enough to keep it ~~alove~~ alive. Some of the ideals we had then really did creep into the real world, and the world is better for it.

Living in the eternal now

2009-07-17T21:42:00.000-07:00

I've been saying this for a long time. Copyright is also a means to keep books (and other materials) off the market until they become irrelevant.

I don't think anybody is immune to the irony of seeing Orwell's 1984 remotely removed from their kindle book reader, however.

Perhaps knowing that this book is freely available in Russia might give some pause.

Re-reading the history of the copyright debate, in, say, 1906, is also useful.

Hell in Honduras

2009-07-05T21:05:00.000-07:00

The closest thing to a reliable internal viewpoint I have found about the mess in honduras is coming from Figgylicious. She writes:

OK people, even if Micheletti not letting him in is a TERRIBLE idea, Zelaya has thrown aside the advice of the international community, the church, his OWN DAMN PEOPLE to not return to the country and he is STILL trying to come in? He is still urging the protesters to confront the military and storm the airport? really? is that REALLY the way that he wants to return? after a bloodbath?

Again--GET YOUR HEAD ON STRAIGHT. Leave! Go to El Salvador! Have some TALKS, for God's sake, and stop panicking the hell out of your people. Jesus. This is complete insanity.

I can still hear A plane going around. Don't know if it's THE plane. No one is saying if he's left for good or not. Good lord.

But really, Zelaya is an irresponsible, thoughtless man. Unbelievable. I do hope that the international community looks at what Zelaya could have occasioned and don't hold him as blameless as they have before. He was calling for civil unrest! Micheletti was wrong, but Zelaya was just as wrong. Unbelievable.

And the absolute best news source, for me, has been the wikipedia talk page on the wikipedia article.

A friend I have, vacationing in Honduras, writes:

So here I am… Finally living the dream that I have had for so long now, and hoping that its not going to be taken away. It is Sun night, time for bbq’s and snorkel tests for my two friends. A nice gathering of people and a great time. As we ready to fire up the grills, my hopes for the evening diminish. 6:30 curfew, all news has been shut down, Nic is supposedly set to invade Hon if necessary. Is the president even still alive? To put it quite simply, shit is hitting the fan and we cannot even find out what is going on. We do not know if we are about to be in a war of some kind, we do not know anything. Yes, I am here on this wonderful little island with great people, realizing my dream of working as a dive professional, but am I going to be able to stay here??? Where would I go, what would I do. I have to start thinking about registering my passport with the american embassy, really?? I come up from diving a wreck, learning how to go inside and all these amazing things, and in the next moment of time, I am thinking of how this really could affect me. What happens if I cannot finish my course?? Financially what are my options, not many I can tell you that. I don’t even have someone I can call and say, hey lend me some money I am in central america and a war just broke out, can you help me?? Yesterday I (attempted) to celebrate the 4th of July, celebrate freedom… in a country that we have absolutely not a single right at the moment. The military is free to do whatever they want and we have to comply. Until this point, it has been in my head a little bit, but now we don’t even know what kind of crazy shit we are going to wake up to. Frustration is running high right now, we have not been able to go out past 10 pm for a week now, and tonight we cannot even be out past dark. People are not coming to the island like they should be, so many places are empty, and most people are talking of leaving. This is such a strange thing to go through having grown up in the states, we have generally had a strong sense of security and although it has been shaken and tested, all has been generally safe and sound. I have never had the military tell me I couldn’t go out or I would be arrested. I have never had to think of fleeing a country because whatever crazy thing might happen. I wanted to be a world traveler, well here I am, living out a piece of history whether I like it or not.

Capitalizing on the Concorde

2009-07-05T10:02:00.000-07:00

This article is the second (out of four) in replies to Chip's arguments in this blog post of mine, as well as this one. My first reply, attempting to combat some of his futilitarianism, is here. (I miss usenet, it was far more suitable to long, winding conversational threads, but I digress)

Chip wrote:

Things like the Concorde, and its ilk are basically (like today's 'globalism) accidents of cheap non-renewable and polluting energy in the form of oil and coal. Today's standard of living, with all its problems is temporary.

Yes, today's standard of living is temporary. It may well be that humanity as a whole has peaked, and is headed towards a downward spiral, or it may still be on an upward one. One thing I am sure of is that the standard of living will change. It may get better in China, and worse in America. Or both systems decline, and South America and Africa see their day in the sun. Secondly I disagree with the concept of a "standard" of living entirely. Does it mean two cars in every garage, a 50 inch tv, and sushi every night? A "standard" implies that there is a universal evaluation of the valued items of living itself. I, personally, am a million times happier now that I no longer have to work 18 months out of the year to keep a roof over my head. There are plenty of things I miss, however. I'd like to get some more advanced vehicle in my house than sandals, and could use an x-ray on my knee. And I wouldn't mind if I could find some agency willing to fund some of the things I think are important for the survival of the species.

Chip then wrote:

Non-renewable and polluting energy is remarkably handy stuff, and we've squandered it, and are continuing to squander it. If you are going to use it, then use it if the your life, and the lives of everyone you know and love and care for including the generations yet to come depended you not wasting it. The Concorde was an interesting experiment. But it was also a huge waste of resources, energy and time [bold mine] .

I heartily disagree with your last two sentences. For starters, an "experiment" is an experiment. By definition, you don't know what the results are going to be.

At the time of Concorde's development, no-one knew the full extent of our oil and gas resources. We still don't, although current surveys appear compelling. We didn't know, when Concorde was started, how efficient such an airliner could be. We couldn't forecast demand accurately, and we didn't know the extent to which sonic booms would inflame the populace against overland crossings. These are some of the direct results of the supersonic experiments. More:

Concorde pushed Duralumin based construction to its limit. More use of titanium would have helped, but in the 1960s the state of the art in use of that material only existed in the SR-71 Blackbird. Over the past 30+ years the use of titanium, has gone from that very specialized use into laptops, and cars, and a variety of everyday devices, including pens. The upcoming Dreamliner aircraft (as well as SpaceShipOne, Two, and Three) make extensive use of composite materials to further reduce weight and fuel consumption.

When operating Concorde at its design point at Mach 2, it was the world's most efficient jet engine.

The amount of fuel used by all the supersonic aircraft in history is a drop in the bucket compared to our "normal" energy uses.

Concorde travelled, per passenger, 17 miles (27 km) for each imperial gallon of fuel — 17 miles per imperial gallon (17 L/100 km; 14 mpg-US). This efficiency is comparable to a Gulfstream G550 business jet (16 miles per US gallon (15 L/100 km; 19 mpg-imp) per passenger), but much less efficient than a Boeing 747-400 (91 miles per US gallon (2.6 L/100 km; 109 mpg-imp) per passenger).

The Dreamliner is expected to be 20% more efficient than the 747.

Research continues into supersonic flight. Aerion has 3 billion dollars in pre-order sales on it's supersonic business jet.

Discarding an idea because it didn't work the first time is a false efficiency. I believe in constant, incremental change, and experiments to determine future directions for that change.

Research also continues into hypersonic flight, as well as into aircraft that emit a more minimal sonic boom.

[I will have more to write about this later, I am just trying to break a very long email reply into more sane component parts right now]

Side benefits of the universal information system

2009-07-04T10:24:00.000-07:00

"I foresee a universal information system (UIS), which will give everyone access at any given moment to the contents of any book that has ever been published or any magazine or any fact. The UIS will have individual miniature-computer terminals, central control points for the flood of information, and communication channels incorporating thousands of artificial communications from satellites, cables, and laser lines. Even the partial realization of the UIS will profoundly affect every person, his leisure activities, and his intellectual and artistic development. ...But the true historic role of the UIS will be to break down the barriers to the exchange of information among countries and people."

Andre Sakharov

1974

I don't think Andrei Sakharov got it entirely right, but man, did the internet come close to fulfilling his prediction!

I keep wondering what was in the ellipsis between "development. ...But the true historic role" - but that information is not on the Internet, yet. Books - at least the ones still covered by copyright, and those long out of print, but still covered by copyright - are only readily accessible via amazon, or not at all.

Babelfish was an early start at breaking down those barriers of language and culture, and google translate is taking that to a whole new level.

New forms of media have arisen, usenet, blogging, twitter, facebook... People can run simulations of the Hubble repair on their desktops, crowdsourcing has become popular where people routinely collaborate, in real time, all around the planet, from richest country to the poorest...

And yet, so many problems are still with us. Do I think they are intractable? No! If I didn't think that we could solve all the problems remaining for humanity, in part, via better technology, I'd be really depressed all the time, instead of just some of the time.

Most of my last week or three of blogging (A New Grand Tour of the Solar System,Sunday Space grabbag,Big Waves, Bad Knees) have been, indirectly, in response to what Chip commented on a blog post of mine last month.

By your own words, you either dismiss or outright refuse to read certain works because they are not 'happy'.

His point stung. I DO try to read multiple viewpoints but sometimes fall into a rut of reading stuff that re-inforces my pre-existing opinions. So I read his links - Kunstler, about the cluster-f*** nation, and Orlov, about the effects of Peak Oil and definancialization...

I was depressed for days.

Yet, my reaction, my self-trained reaction, of looking for an opposing viewpoint, and exploring the history of the ideas and predictions, finally kicked in... I went and researched Kunzler, and found he'd made specific predictions that turned out to be wrong, so far. It doesn't mean that he's wrong on everything, it just means he isn't God. I completely agree with Kunzler about the core problem - today's civilization relies on cheap energy, and it is running out. His solutions are interesting, and no doubt there are others that both agree with (both of us) and are trying to solve the problem.

So, Chipper - I have a suggestion - go read someone optimistic for some balance, and some optimism, once in a while. I still find Buckminster Fuller comforting...

I have spent the last days being relentlessly optimistic, and arguing with people that I think are making the wrong decisions, or doing the wrong thing over and over again, because of habit.

Yesterday I ran into a lady with a BS in Social work, who was taking a quick two week tour of Central America before returning to America to study for a Masters. Getting more education is a worthy goal, but her reasoning was flawed - there were no jobs for her existing background and school was all she knew how to do. She had no debt but was preparing to take on a lot of it to get her Masters... I strongly encouraged her to continue her wanderjahr, if she could, find something that she loved to do, that paid, and stay out of debt....

Last week I also went and reviewed the current state of the climate change debate, and found no reason to change my opinion that waiting for more data to come in was the right thing. I am especially looking forward to Anthony Watt's report on the effects of bad siting for temperature measurement on the global warming average.

And to balance out the bad news in Honduras...

I read about solar power satellites being funded, and about improving tuna aquaculture, about the accomplishments of legendary engineer Isambard Brunel, and the Age of discovery... and drooled over the upcoming model S all electric sedan, from Tesla Motors. After witnessing 10s of billions of dollars poured down the ~~rathole~~ bailout of America's failed carmakers, seeing Tesla - an American company, based in California, a company trying to build cars people actually want, after being forced to sell part of themselves to Daimler - finally get a loan sufficient to build the plant to create that car, in California - really cheered me up.

And - thanks to the universal information system, predicted, more or less accurately, by Sakharov, in 1974, I was content to believe, once again, that somehow, we'll muddle through.

A New "Grand Tour" of the Solar System

2009-07-03T16:42:00.000-07:00

What would it take to resurrect something like the long-canceled Hera mission, expanded and revised to support a "Grand Tour" of the nearby asteroids and comets? Hera was intended to visit 3 asteroids in the 3-5 years after launch, and then return to earth. The preliminary analysis for that mission (in 2004!) showed 60 possible interesting trajectories and candidates for exploration.

The detection of new Near Earth Asteroids (NEOs) continues at a rapid pace, a pace which will increase upon completion of the LSST in 2015. The number of NEOs known has at least doubled, perhaps tripled, since 2003. (cite needed, empirical evidence supplied below)

What I have in mind is a series of small spacecraft, say, 4-12 in number, that would each visit 3 asteroids or comets over a 4-6 year period of life. The tour is only "grand" in that we could explore nearly every known asteroid classification, and would probably be considerably cheaper in current dollars than Voyager 1 and 2 were, particularly if a new launcher like the Falcon 1 or 9 was used.

I haven't the foggiest idea how to generate the enthusiasm for this idea, or the funding, aside from writing about it, and... perhaps... since my stock in trade is as a software engineer, maybe I could work towards making broadly available the software for calculating possible courses (trajectories). Perhaps being able to plot a real course for Cruithne, or tens of thousands of other small bodies, like Sulu from Star Trek, would get more people interested and involved. I know the simulations that Bruce Damer did of the Mars Rover were wildly popular, particularly among youth.

All I really know (thus far) about re-solving this problem is from a chat with one of the scientists involved (items in bold are my open questions, italics is what he told me):

The trajectory code used for that analysis was JPL's Midas patched conic trajectory tool.(how does a US citizen get access to Midas? The conic section tool appears to be a commercial product from JPL. Is there an alternative? Is it even necessary?) The tool was automated to run 1000's of combinations of solutions. (How? What happened to the code?) These solutions were reduced using impulsive delta-V as a primary FOM. (OK, that's the easy part) The solutions that filtered to the top were then run through a low thrust trajectory code, segment by segment, to generate a end-to-end low thrust trajectory profile. (Solar-Electric propulsion makes a lot of sense, but old fashioned chemical propulsion might be more sustainable for in-situ refueling, being able to simulate a wider variety of spacecraft (included manned ones) would be useful)

Unfortunately it was rather labor intensive process and the work did not continue. Sigh.

It has taken me a long time to get interested in space again, ever since Trailblazer became ashes over the Pacific. What is making me think about it is that orbit@home is now up and running, and there is an amazingly powerful n-body code out there for CUDA , as well. Perhaps this would make it possible to solve a "New Grand Tour" problem for large numbers of asteroid and comet targets using differing types of spacecraft. For all I know, a 200 dollar card with CUDA and suitable software may well be more powerful than the compute clusters used during the development of Hera. (see left for a lovely example simulation of whole galaxies in collision - surely something like that ought to be able to help plot a few courses in our piddly little solar system?)

In terms of delta-v: there are presently 952 good reasons to go to the asteroids rather than the Moon. Some delta-v reasons are almost twice as good than the moon option.

It's also worth repeating the Deep Impact or Don Quijote missions 3756 times by the same criteria....

2 years ago (more accurately, June 25, 2007), in terms of delta-v, there were only 536 good reasons for the asteroids rendezvous and 1920 for flyby missions.

PS: I note that estimated delta-v via shoemakers method is not a particularly good criterion for justifying asteroid missions over Moon and Mars missions, but it may provide a good starting point for a conversation over the resources required to explore the solar system.

PPS: I really don't want to explain delta-v, please see wikipedia for delta-v, interplanetary superhighway, etc....

PPS: I am rather reluctant to use the "Grand Tour" meme (new name suggestions welcomed!), as the original Grand Tour was expensive, and ultimately it was scaled back to the two Voyager missions. Voyager's results were, and continue to be, spectacular. (I really don't like "Grand Tour" as a name - The Grand Tour meme is being reused for the Cassini mission)

I would really like to see the above chart updated, but given what we know know about the solar system, vs what we knew in 1996, it would be all orange and red inside of Jupiter's orbit, on the scale at which the objects are plotted.

Biking that last bit

2009-07-02T10:13:00.000-07:00

Chipper posts some pictures and writes about his biking to work:

Most days it adds about 45 minutes to 'bike' (I ride a trike) the last 10 miles vs what it takes to drive. However, when the traffic is particularly abysmal and noxious, I don't know that it takes me any more time at all. Guess why I don't know? I don't know because I am not stuck in it! I have no earthly idea what is going on out there on the roads. As these pics show, this is how I spend that + or - hour that makes up the last leg of my commute.

I had intended to get my ebike to SJDS last november. I had big plans on exploring the area and ultimately settling out of town somewhere, to live more harmoniously with nature and get out of touch with my inner geek, and back in shape. Regrettably, the airline I flew out on wouldn't let me take it, so it has sat in my parents garage since then, awaiting my return.

I miss my bike. I felt it would have been a lower-maintenance alternative to a horse and far better exercise, and I would have explored a lot more of my local area than I have been able to explore on foot. Perhaps this year I'll get it... or a horse.

From a tourist inside Honduras...

2009-07-01T20:46:00.000-07:00

A friend writes:

We are all watching and waiting, the biggest thing we are upset about is that nic stopped sending us chocolate milk! haha. Kinda kidding...

It is pretty strange, we are cut off so much from everything here but yet in a way we are right in it. Last I heard the borders and all airlines were closed. Everyday we all try to share whatever that daily piece of info is that we get, more like gossip than news. It sounds as though Venezuela is threatening to cut off our oil supply, and that in general shit may hit the fan. There is a level of anxiety that we are dealing with, but the surreality of where we are can overshadow that so easily. I still spend my days diving, studying and lying in hammocks. Best as I can say, it would be best to get Zalaya out of power and for the US and the rest to back the fuck off because they are causing more problems than they are helping. It would be far more dangerous for me to try to leave and go through the mainland than it is for me to stay here. The military enforced curfew is a pain in the ass and very pointless here. I guess that my main point is that yes I am okay here, for the time being. We are starting to feel the ripples here and yes we are starting to get a bit nervous. I do believe that the best thing for me to do is stay here.

More Honduras (and Nicaragua) updates

2009-07-01T11:06:00.000-07:00

Alexis Argüello, the mayor of the capital of Nicaragua, Managua, is dead.

The front page of La Prensa, the chief opposition newspaper in Nicaragua, has been offline since at least 8AM this morning. However, you can do deep linking (eg, http://www.laprensa.com.ni/archivo/2009/julio/01/noticias/portada/) to some pages, for some things. For example, clicking on http://www.laprensa.com.ni/archivo/2009/julio/01/noticias/ultimahora/335994.shtml works, while http://www.laprensa.com.ni/archivo/2009/julio/01/noticias/ultimahora/335991.shtml returns an error in even connecting to the website.

This, to me, is a sure sign of clumsy censorship or a DoS attack against La Prensa within Nicaragua that is taking place as I write.

The subject of that apparently blocked article and precís are:

Asesor no notó nada extraño en Argüello

Un asesor del alcalde de Managua, Alexis Argüello, aseguró esta mañana que estuvo anoche con él y que no percibió que éste tuviera algún problema emocional. Según los primeros informes extraoficiales, el tres veces campeón mundial de boxeo, se suicidó de un disparo en el pecho poco antes de la dos de la madrugada. "

which, translated, is:

Adviser did not notice anything strange about Argüello

An adviser to the mayor of Managua, Alexis Arguello, said this morning that last night was with him and that he had not felt any emotional problems. According to early unofficial reports, the three times world boxing champion, committed suicide by a shot in the chest shortly before the two in the morning.

The current article, also about Argüello, http://www.laprensa.com.ni/archivo/2009/julio/01/noticias/ultimahora/336012.shtml, is also blocked.

News of Alexis Argüello's death has not made the other major newspaper El Nuevo Diario, as yet.

Update: 12:17 PM OK, Nuevo Diario just ran the story (google translation here)

More on this and the Honduran situation in a bit.... this is what I wrote about the crisis over the last few days.

Update 2:15 PM OK, whatever was blocking parts of La Prensa appears to have been lifted.

Constitutional Crisis in Honduras

2009-06-29T19:05:00.000-07:00

I really hate the word "coup". Usually it implies a military takeover, sometimes foreign backed, such as the one that took place in Guatamala in 1954, or the one that toppled Honduras's government in 1963.

What appears to have just happened in Honduras - the ouster of highly disliked (25% approval rating) president Manuel Zelaya - by order of the Supreme Court, Congress, AND the military - does not look very similar to that (at least from first appearances)

For starters, he was replaced by the legal line of succession, Roberto Micheletti, the head of congress, and also a member of Zelaya's own party.

But, the word "coup" rules the day, and the rhetoric, and the debate inside wikipedia is astounding.

The rhetoric is being spewed by a peculiar alliance of just about every other politician in the world - left, right, it doesn't matter. It seems to boil down to: just about everybody holding power can agree on one thing: Arresting (and deporting) a president who clearly is attempting to violate his country's constitution, is wrong.

The irony of that observation is what is compelling me to write today - I confess to being amused by watching each ideology put their individual spin on things:

Xinhua (China), The New York Times (USA), Infowars (USA), Rueters (?) Finding Dulcinea, (EU), and Latin America.... (sorry, I will update these links to make more sense when I have time) AP...

Not to mention the irony of USA President Obama coming to the aid of a world leader that called for the legalization of drugs in order to eliminate the narco-trafficing trade... while he hangs out with the Colombian President, whose country is the source of much of that trade.

The best and most detailed reporting as to the depth of the crisis seems to be taking place in the Wikipedia article on the 2009 Honduran Political Crisis, which starts off by quoting the Constitution of Honduras:

Article 239 of the Honduran Constitution, which forbids any former chief executive from being re-elected President, states that any citizen who proposes reforming this law, and any others who support such a person directly or indirectly, are to immediately "cease carrying out" any public office. The Constitution, however, establishes no process for impeaching or removing a president. Furthermore Article 42, Section 5 of the Constitution states that citizenship is lost for "inciting, promoting or supporting the continuation or the reelection of the President of the Republic." According to the same article, revoking citizenship for this reason requires a court sentence and then a government order ("acuerdo gubernativo").

The intended referendum was rejected by Congress, the attorney general, and the top electoral body, and ruled illegal by the Supreme Court, provoking a serious political crisis in the country.[13] The National Congress passed a law[14] forbidding holding referenda less than 180 days before the next general election; as the next elections are set for 29 November 2009, this invalidates the referendum bid. In addition, the Honduran Constitution expressly forbids amendments or reforms altering presidential terms or allowing re-election.[15]

Congress had begun discussing how to impeach Zelaya but lacked a clear constitutional process to do so.[12] Congress, including most of Zelaya's own party, had voted for an urgent investigation of whether Zelaya had violated the constitution and even whether he was "mentally incapable" to hold office.[16]

(I note that the wikipedia article - several hours after I wrote this - bears little resemblance to the article I first linked to. I wish I'd taken a full snapshot of it then. Among other things, it used to have a translation of the referendum in it.The debate inside wikipedia is also very informative.)

The government refused to print Zelaya's referendum, so he got the ballots done in Venezuela.

His chairman of the joint chiefs refused last week to distribute that referendum. (I guess, after decades of dictators for life, you get a little touchy on the subject of term limits...)

Zelaya fired him.

The Supreme court ruled that an illegal act. So did the Honduran congress. The supreme court ordered Zelaya's arrest... and... he was arrested!

You have an angry and divided country, what are you going to do with a president under arrest?

Lock him in a penthouse suite, like Bernie Madoff? Throw him in a common jail with a roomate named Bubba? Kill him? Have him get pardoned immediately by the likes of a Gerald Ford?

Arrest, deportation, and same day release in a neutral country, like Costa Rica, I don't
know if that was legal or not, by Honduras law. I've been trying to figure out the proper procedure for arresting a president for decades now... but they didn't have a procedure in place for impeachment, much less arrest.

Deportation (and the resultant freedom of movement, ability to organise, and mouth off to the press) seems like a pretty sane alternative to those options, however one with far less than ideal characteristics if you didn't believe that the truth was on your side.

Recent polls show support for Zelaya in Honduras has dropped to around 30 percent in recent months.

What country takes their constitution so seriously as to actually throw out a scoundrel? One accused of trying to import another constitution entirely?

Why, if more countries took their constitutions seriously, the populace might get ideas about their governments actually respecting them. No wonder the world condemnation!

From this outdated American analysis of the Honduras constitution:

"Honduran constitutions are generally held to have little bearing on Honduran political reality because they are considered aspirations or ideals rather than legal instruments of a working government. The constitution essentially provides for the separation of powers among the three branches of government, but in practice the executive branch generally dominates both the legislative and judicial branches of government".

Not this time, apparently, although an analyst says

the action taken against Mr. Zelaya harkens back to a dark period in the region's history, when military coups were common... today, coups are not seen as acceptable under any circumstance and that international pressure likely will prevail.

What I enjoy most, as usual, is the commentary on the news articles, from ordinary people... In this case, running overwhelmingly in favor of the action:

"I am a Honduran citizen who feels extremely proud of the measures take by National Congress, The Supreme Court of Justice and our military forces. Mr. Zelaya had been warned many times that his actions were breaking laws of the constitution. There has been no "coup" in Honduras. Military Forces were simply following orders from the Supreme Court of Justice because Mr. Zelaya refused to back down from his plans. The poll is an illegal act benefiting only Zelaya and no one else. Good riddance!"

"I'm a Honduran resident, we are ok, it's true the situation in my country is not the best, but most of the people support the position here, we DON'T WANT Manuel Zelaya as a president again, during his period he has only been manipulating people, blackmailing ppl, threatening to dismiss state employees if they not support him, and many things that just people that we live in Honduras know...please take a look at the real situation not the lies that he says to the world."

"We're relieved that Zelaya was removed from the presidency. We´ve lived weeks of anguish watching how the ex president of Honduras acting as he was above the law, wondering what would happen to our democracy and to our country. If the President of Honduras thinks that he is above the Constitution, then what happens to the rest of the leaders? Are they above the law as well? Democracy has won, and the intl. community has to listen to the will of the people of Honduras. Don’t victimize Zelaya!"

How this plays out is going to be very interesting.

Update 8AM, Jun 30:

I am not going to change the text of what I wrote last night, although I may go back and clean up a few phrases and find more cites today, in addition to commenting further on this page. I find history changing under my fingers, as what happened in wikipedia in the last 24 hours, somewhat disturbing. The process by which wikipedia goes about that is transparent, open, and fascinating... and like wikipedia, I believe that "when the facts change, so does my opinion", but, in part, my blog is my journal of what I thought, when. Editing what I already wrote does not fit the wikipedia model in that respect.

Although I make it clear in the article and links above that I'm not particularly fond of Zelaya, or the CIA, or coups, or the ideological press, or the treatment of Madoff, and quite a few other things, I would like to re-iterate my main points were, 1) The multiple ironies involved and 2) the role of spin, and (indirectly) meme shifts, which I wrote about extensively in the last American election.

1) Irony

A comment (by a Honduran college student) on the wikipedia talk page put it best:

There should had been a lawful process to kick Zelaya. Without it, we lost the favour of EVERYONE in the world that doesn't live in Honduras. That means we are open to the invasion of nondemocratic countries that have close ties with Zelaya (understand Venezuela). BUT! With the help of every other country. So, democratic people (USA, Mexico, France, etc) will be fighting a democratic war against people that wanted to maintain democracy, and will reinstitute a anti-democratic leader.

Final statement: The problem is that it is easy to recognize armed assaults on the Rule of Law. But, when this Rule of Law is attacked without arms? When it's attacked with corruption, with helding of budget, corruption at the ballots (I know first hand, just, if I said something, they would kill me), and continuism? Why doesn't anybody recognize that, although the FORM of the coup was completely wrong, the REASON of the coup was completely right? Why would they support a unlawful president?"

2) The role of the media, and "spin"

Two competing memes are in play. 1) "coup" - and 2) "the referendum was about allowing the president to be re-elected"

Most newspapers reports have converged on these two memes as shorthand.

The Honduran government disputes the first, as do some bloggers inside the country (even if no-one else outside the country does, 24 hours later), and the second was not the text of the referendum. From that always helpful wikipedia talk page...

“¿Está usted de acuerdo que en las elecciones generales de noviembre de 2009 se instale una cuarta urna para decidir sobre la convocatoria a una Asamblea Nacional Constituyente que apruebe una nueva constitución política?”

Translated: "Do you agree for a Fourth Urn to be installed on the November 2009 general elections in order to decide whether to call or not for a National Constitutional Convention to approve a new Political Constitution?" -- so the question is not: Do you want the current president to be re-elected?

It is interesting to compare DailyKos's (A progressive-liberal) views of
Honduras - and Iran. This article, although not representative of dailykos's views as a whole, was interesting, as were the comments.

More on this in a bit, I need a few more cites from more varying perspectives.

Update: Jun 30, 5:00 PM I haven't had power all day, a major storm hit this morning.....

Update: Midnight A lot has happened today, everything from the UN voting unaminously to endorse Zelaya's continued presidency to multiple protests on both sides, to me losing contact with the one friend I have in Honduras for the last 24 hours, entirely.

Probably the most elequent person inside Tegucigalpa, in her own way, was Figgylicious yelling back at the TV. I keep wondering when or if the MSM will try and follow up on the thousands of blog writers and commenters I've seen typing their hearts out.

Sunday Space Grabbag

2009-06-28T08:38:00.000-07:00

I haven't been writing about space a lot recently. Progress in that field is slow, so I figure that only writing about it once every 3-6 months will suffice. After September it looks to be an exciting time in space again, with the Falcon 9 scheduled to fly, as is a primitive Ares. Rumor has it that SpaceShipTwo will be in drop testing by then too....

This July should be good, with the 5th Falcon scheduled to fly sometime then.

If you prefer to be updated more regularly I highly recommend subscribing to the arocket mailing list, and adding a rss feed or 10 from the blogroll of The chairforce engineer.

Presently there is a lot of political manoeuvring around finding an alternative to Bush's VSE that the latest crop of politicians can put their stamp on. Much seems up for grabs, ranging from outright cancellation of Ares-1, to extending shuttle life, to swapping engines to the SSME in the VSE, to refocusing on getting to Mars via Phobos or the asteroids and comets.

SpaceX's Falcon 9 continues to make steady progress. 6 of the engines have checked out, the first Dragon capsule has been built, and Elon musk has been talking up his story in front of the Augustine commission. (video here)

1) Mike Griffin's departure from NASA occurred 6 months back, and a replacement, Charles Bolden was named. Although the two next-gen R&D projects (Constellation and Ares) are a mess, the rest of NASA is functioning as well as it has in a long time - All Griffin's shuttle missions were great successes. The magnificent Hubble repair mission puts a final feather in the cap of Griffin's NASA, with a second shuttle, ready on the pad for a rescue mission, also an impressive feat. Numerous instrument missions were launched without a hitch, even including Dawn, which was flying with nearly obsolescent hardware. The COTS program, also, appears to be successful.

(For the record, I opposed both the Dawn and the Hubble missions)

Griffin, the technologist, was better than his bean-counting predecessor by a country kilometer. Let's hope his shuttle pilot successor can navigate Congress and future NASA R&D as well as Griffin got our existing space assets to LEO.

2) Buzz Aldrin has been a very busy ex-astronaut lately, publishing a new book, promoting a different vision for space exploration, and even recording a hip-hop song.

Aldrin concludes: "But for this dream to happen, NASA needs to dramatically change its ways. Its myopic Vision for Space Exploration will never get us to Mars. Progressive innovation and enlightened international cooperation will. President Obama and Congress need to set NASA right - and soon."

Aldrin's separate vision for space exploration dovetails more closely with mine (or rather, mine dovetails with his).

3) China - it would take more time today than I have to write about China... try this for updates. They are going to Mars this year, and plan to launch over fifteen satellites this year, too. Aldrin seems far more concerned with Russia than China, for some reason...

4) Europe's ESA launched Herschel, and the first images came back recently, the resolution (right) appears far superior to the American Spitzer telescope.

5) I have been patiently waiting to see if more Apohele asteroids would be confirmed. They are very difficult to detect using our existing methods. In addition to their potential hazard, they appear to be easy to reach from a delta-v perspective, and given their distance from the sun, exploration using solar powered devices appears cheap and effective, although heat is a problem - 163693 Atira, a 2km in diameter asteroid, apparently has a temperature of 323 Kelvin (50 Celsius, 122 degrees F)!

I'd like to see an instrument mission launched, call it: the Inner Solar System Explorer (ISSE), which would look for and at asteroids and dead comets in the region between Venus and Earth, looking back at space from well within Earth's orbit.

Perhaps it would be able to get close-up looks at a few of the hundreds of rocks passing through.

5) Asteroid 2008 TC3 broke up over Africa as expected, but it took a while to find successful pictures of the re-entry. I was hoping we would get more data, but the fragments were found, strung out in a line, by a bunch of volunteers....

Update: While researching more possible points for this blog entry I ran across the Columbia Crew Survival Report.
Given my emotional state at the time, I hadn't bothered to read it when it came out.

I just finished reading it.

The report was a model of clarity, a tour de force of analysis, showing clear advances in the state of the art in debris recovery, computer modeling, and tracking, since the Challenger accident.

Emotionally, I did fine, until I hit the last (400th) page of the report, which had the following logo, which I had never seen before that moment.

AD ASTRA PER ASPERA * SEMPER EXPLORO
To the Stars, through difficulties * Always Exploring

Then I had one last good cry of grief, and determination.