alex's blog

The art of not losing things

My wife is better at finding things than I am. I am better at not losing things than she is. It's an important difference.

Being good at 'finding things' means she's more careful when she goes place to place looking for something. She notices details I don't. She thinks of places to look which don't occur to me.

Being good at 'not losing' things means I tend to put stuff back in the same place every time. Sara is constantly losing her keys, purse, or cell phone because she tends to drop them wherever, then go looking for them later. I tend to put stuff back in one place. (Notice I say *tend to*. I'm definitely not claiming any perfection here. I need her help plenty of times finding stuff I've stupidly dropped in some random place...) I developed the habit because, as mentioned above, I'm not very good at looking for things. This is my way of working around my own lack of search skill.

Whenever I have something in my hands which I no longer want to be holding, I tend to ask the question "where is the first place I'd look for this thing?" Whichever place first comes to mind is the place where I put the thing. Not perfect, but a lot better that before, when I combined the 'drop stuff anywhere' storage strategy and the 'carelessly and unproductively scan for things' search algorithm.

Why does this matter in the context of a blog about coding? I think the same rules about 'where to put things' apply really well to software or database design. 'How do I want to search for this?' should play a large part in 'how should I store this data?'.

Of course there are plenty of other concerns. For example, tailoring a database schema too closely to a single kind of search often reduces the ease with which you can perform other kinds of searches of that data. But 'how will I search for this stuff' should be *part* of the design process. And, if you can't answer that question at all, I think it's worth asking 'do I really need to store this data at all?'

Opting Out of Verizon's DNS Services

Got weird error trying to access www.google.com.

No Googles!

Co-worker helped me figure out this was Verizon's 'DNS Assistance' messing up.
http://www.verizon.net/central/vzc.portal?_nfpb=true&_pageLabel=vzc_help...

Tried their 'opt out' option. Visiting a non-existent domain brings up an incredibly lame error page. The page you see below is HTML copy/pasted from the standard Internet Explorer page (including image references to 'C:\...'), with some Verizon links slapped on the end of it. Wow.

Verizon DNS Assistance Fail.

What the heck?

I switched to 4.2.2.1 and 4.2.2.2, but there are plenty of other public DNS providers. This is kinda funny since 4.2.2.1 is also a Verizon server, but it doesn't have this annoying 'helpful' feature.

http://www.tech-faq.com/public-dns-servers.shtml


Our Smart President

Just read a story about the Obama administration using the Drupal CMS for whitehouse.gov. Maybe we can say they're learning from us here at Deanspot?

http://techpresident.com/blog-entry/whitehousegov-goes-drupal

http://www.drupal.org

DRY? How about DROP?

The acronym DRY is popular with Rails folks, and maybe elsewhere. Don't Repeat Yourself. If you see two sections of your code doing essentially the same thing, create a single method which is called from those two places. Less code, less maintenance, better test coverage, less chance for errors later on. In database terminology, I'd talk about eliminating redundancy and avoiding update anomalies. It's the same thing. It's great advice, and clearly captures a bit of wisdom that (I'd venture to say) is applicable to most any kind of programming.

In addition to not repeating yourself, though, how about Don't Repeat Other People? Open source is all about scratching your own itch, so if you're driven to solve an old problem in a new way, go for it. But think first about why you're doing it.

Consistent Tab Navigation in OSX

Tab navigation is one area where the OSX user experience seriously lacks consistency. It seems like every application implements a different keyboard shortcut for 'next tab' and 'previous tab'. I am a person who uses lots of tabs in many different programs, and I prefer to do as much as possible from the keyboard. Reaching back and forth from keyboard to mouse is a pain. The fact that it's a different key combination in every application is a serious annoyance.

I've taken things into my own hands and tried to hack some consistency out of this keyboard jungle.

Compiling Ganglia gmond and gmetad on OSX 10.5

Ganglia is a set of daemons used for performance monitoring. It works on OSX, but getting it running on that platform can be a bit touchy. Here's the series of steps which have worked for me recently.

Note : I'm writing this for those who already knows what Ganglia is, and want to get it running on OSX. There are lots of other articles out there describing Ganglia, what it's used for, and offering more detailed configuration information.

Looking for Rails help?

Well, if you're reading this, you probably know me, so just ask me directly. Otherwise, try Rails Mentors, a new site that's just been launched to connect experienced and newer programmers.

Posting to Drupal from an iPhone via SMS

Why?

Why do I want to do this? The iPhone allows you to send regular emails, and the MailHandler Drupal module already allows you to submit content via email. This is about 90% of what I want. I'm going to be driving cross-country, and in many places I'll have phone reception but no data service. SMS requires only normal cell reception, not Edge/3G/etc data coverage, so this is pretty important for me.

Why don't I just use Twitter? That's harder to answer. Mainly, DIY is more fun.

So, all objections being skillfully silenced, here's what I did...

Programming & Professionalism

The post below is a response to a talk given at the recent RailsConf, about professionalism and programming. There's a bit of profanity, so don't read if you're easily offended. If you don't have any such unreasonable hangups, I think there are a lot of excellent ideas here.

http://gilesbowkett.blogspot.com/2009/05/what-killed-smalltalk-my-balls....

Making Donuts

External Hard Drive for Sale

http://phoenix.craigslist.org/evl/sys/1104192621.html

Free delivery for family and friends!

Sold! To the man in the funny hat. Yes, you sir, you in the back.

Easy way to list public methods for a Ruby object

I often have new code I'm trying to figure out, and constantly jumping back and forth between a console and a web browser with documentation sometimes really slows me down. Often times it's easy enough to remember which method you want just by seeing a list, even without detailed descriptions.

Any object has a public_methods method, which returns an array of strings listing the commands. Trouble is, all the methods it inherits are of course listed in there, and Object provides a LOT of stuff. It's easy for the interesting stuff about the instance at hand to get lost in that noise.

So, I'm trying a little experiment. I added this snippet to my ~/.irbrc file. Works on Linux and OSX, and in either normal irb or in the Rails console.

class Object
  # print public methods which are not inherited from Object
  def pm
    (self.public_methods - Object.public_methods).sort
  end
end
Now any object has a pm method. So what?

Loading development environment (Rails 2.2.2)
>> class Blah
>>   def foo; end
>>   def bar; end
>>   def blatz; end
>> end
=> nil
>> b = Blah.new
=> #<Blah:0x12b6b04>
>> b.pm
=> ["bar", "blatz", "foo"]
>> b.public_methods
=> ["returning", "to_yaml", "pretty_print_cycle", "inspect", "method_exists?", "stubs", "to_param", "extend_with_included_modules_from", "subclasses_of", "require_or_load", "clone", "method", "to_enum", "to_yaml_properties", "public_methods", "__metaclass__", "to_json", "suppress", "instance_variable_defined?", "instance_variable_names", "dclone", "equal?", "blatz", "freeze", "expects", "with_options", "foo", "methods", "respond_to?", "geometry_data_types", "instance_exec", "enable_warnings", "to_matcher", "to_query", "silence_warnings", "reset_mocha", "to_c", "dup", "enum_for", "instance_variables", "__id__", "copy_instance_variables_from", "duplicable?", "eql?", "object_id", "pretty_inspect", "require", "id", "send", "singleton_methods", "silence_stderr", "encode64", "class_eval", "taint", "taguri", "require_association", "stubba_method", "instance_variable_get", "frozen?", "instance_of?", "__send__", "require_library_or_gem", "b64encode", "to_a", "pretty_print", "taguri=", "daemonize", "remove_subclasses_of", "`", "type", "debugger", "blank?", "instance_eval", "protected_methods", "display", "==", "silence_stream", "unloadable", "decode64", "===", "acts_like?", "pm", "c", "to_yaml_style", "instance_variable_set", "extend", "kind_of?", "to_s", "extended_by", "class", "hash", "breakpoint", "present?", "mocha", "private_methods", "=~", "tainted?", "decode_b", "mocha_inspect", "instance_values", "untaint", "nil?", "load_with_new_constant_marking", "__is_a__", "bar", "stubba_object", "pretty_print_inspect", "require_dependency", "is_a?", "pretty_print_instance_variables", "metaclass"]

See the difference? Call it micro-documentation.

Installing Apple's calendarserver on Ubuntu

Update : 12 Dec 2009

I've added a new post describing how to install Darwin Calendar Server (aka DCS, aka Apple's Calendar Server) from source on Ubuntu 9.10.


I've wanted to have a shared family calendar for a while. We've tried paper calendars. They never got updated, and moving things around was a pain. Sara may disagree with me, she still prefers a regular paper datebook. But the ability to remind each other of when various things are scheduled is really better in the electronic world. My work uses calendarserver (part of OSX Server) to schedule meetings, and combined with a client program like iCal, it's a great system. Just because it's Apple, though, doesn't mean it's only available for Mac users. Programs like Outlook for Windows or Evolution for linux can also subscribe to calendars published by calendarserver. You can find a partial list of compatible clients at http://trac.calendarserver.org/wiki/CalendarClients, but any program which can speak CalDAV should work fine.

CalendarServer is, itself, freely available. You don't need a Mac server, and you don't have to buy OSX. It's written in Python, so it runs on pretty much any platform, and is distributed under an MIT license. It's even available via apt in recent Ubuntu releases, which makes it really really easy to set up if you have an Ubuntu machine handy. These are some quick notes about how I configured this for our local network.

Monitor resource overruns in OpenVZ

OpenVZ is a great virtualization tool for Linux servers. I've repeatedly run up against various resource limits, which can sometimes lead to really weird errors like 'cannot allocate memory' when you do something awful like ls -l. I cooked up the following script to keep a log file of times when a server overruns its bounds. I can then either raise the limits, or try to correlate the overrun time to something going on at that time. Read more for the full details.

Syntax highlighting problem with .html.erb files in Kate text editor

Recently, I've been trying out the Kate (http://kate-editor.org/) text editor when programming. It's free, lightweight, and seems quite nice. It has most of the features of TextMate (my current favorite), runs on Linux, and is free.

Last weekend I noticed the syntax highlighting for Ruby on Rails view templates (files ending in .html.erb) was all messed up. Matching tags were not marked, functions weren't highlighted, and everything was basically really difficult to read.

OSX clipboard from a Terminal

Sometimes I want to get something into the system clipboard without using a mouse, like wanting the entire contents of a file in the clipboard, but without the odd linebreaks that text editors are prone to inserting all on their own.

I discovered the 'pbcopy' and 'pbpaste' tools, which are just perfect.

$ echo "blah" | pbcopy

Now "blah" is in my clipboard. I can Command-V in any window, and there it is.

I can also manipulate the contents of the clipboard in any script with 'pbpaste'

$ pbpaste | wc -l

I just counted the number of lines of text in the clipboard. Yay!

http://www.commandlinefu.com/commands/view/751/mac-os-x-copy-and-paste-t...

Oh God not this again...

I really enjoy being part of the Phoenix Linux Users Group (PLUG). It's a group of smart, friendly, helpful folks who love free software and solving technical problems.

Some, unfortunately, also love to bitch about Microsoft and spew gossip and conspiracy theories with some regularity. Seems everything Microsoft does is pure evil, and the heroes of freedom need to oppose them at every turn!

The 'us vs. them' mentality seems quite childish to me. I don't have any love for Microsoft, but my interest in free software is about what it IS, not what it ISN'T. If Microsoft wants to lock things down, invade privacy, and charge a lot of money for it, I'll continue to not use their products and advise others to do the same. That doesn't automatically mean everything they come up with is existentially wrong.

There's a lot more I could say on this topic, but instead I'll finish with a blatantly unrelated cute picture to show how great it is to share.

DSC01753


Test::Unit gotchas

I've run across a few things worth remembering when writing unit tests for Ruby using Test::Unit. Might be obvious to some, but they tripped me up once or twice.

class SomeTests < Test::Unit::TestCase
  def test_something_happened
    puts 'A'
  end

  def test_something_happened
    puts 'B'
  end
end

Because you're allowed to modify methods at runtime, you get 'B' in your test output, and the first test_something_happened is never run. Shouldn't usually be a problem, but this is one of those times I wish you could make Ruby return an error, or at least a warning, when methods are redefined. If 2 tests get the same name, you could have a broken test with no warnings.

def test_load!_throws_exception
  assert_raises SomeException do
    Record.load!( invalid_record )
  end
end

Apparently, you can name a method 'load!', but you can't include '!' in the name of a test method. test_load!_throws_exception is never run, and no errors are generated. When I rename the method to test_load_bang_throws_exception, then the test actually gets run.

Running your test case in this way :

  $ ruby -w2 test/unit/some_test.rb

will at least get you a warning, but it's not the default. It would be nice if the Rails rake tasks enabled warnings like that when running 'rake test'.

Lots of little methods?

The consensus I hear in OO design is to prefer lots of little methods over big monolithic do-everything methods. This makes plenty of sense to me. Little methods are easier to re-use and easier to test. The additional claim that they are easier to understand has been a sticking point for me. I don't necessarily agree that's true.

In terms of just 'what does this code do', it's certainly easier to understand what 5 lines of code as opposed to 50 lines. But in terms of 'why would you want to do that', I don't think there's an obvious correlation between length and comprehensibility. An API with hundreds of tiny methods isn't simple, and isn't necessarily any easier to understand than the same lines of code grouped into 10 functions. If anything, it can be MORE complex to understand, because you probably have to call several of these little methods (and probably call them in a specific order) to do anything useful.

That being said, I still think lots of little methods are the way to go. I would also like to see some higher level methods, which don't do much more that call sets of more fundamental methods. This allows a new user of the API to see not just WHAT is done, but also WHY. Answer the 'who cares' question. I think to some extent there's an inverse relationship between comprehensibility and the number of options you are presented with. The more choices you have, the more you have to examine them all to figure out what's the right thing to do. Give me lots of little methods, but also give me those higher-level do-something-useful methods.

Not happy with Active Record today.

UPDATE 1/13/2009
I ran across an article on InfoQ about Performance Anti-Patterns in Database-Driven Applications. Check out the section titled 'Load More Data Then Needed'. It's very relevant to my beef with ActiveRecord, where 'SELECT *' is the default.


Using :includes always uses OUTER LEFT joins, even when it really should use INNER joins. No way to configure or change that. Also ignores your :select clause, so if you only want a few fields from the dependent models, too bad! This stinks when you want a text field from a dependent model, and you're forced to bring a giant GeoRuby polygon along for the ride.

Using :joins does INNER joins, which is great. But if you later reference a dependent model, AR goes back to the db for 'SELECT * FROM dependents', even if you've selected data from that table in your original :select.

So there just doesn't seem to be a way to
* Load dependent models with INNER joins
* and only load a few fields for those models.

My 'rant in a comment' from the source code I was fighting looks like this...

@locations = AlertLocation.user_id( current_user.id ).find(
  :all,
  # :select=>'alert_locations.*, us_counties.name, us_zones.name, us_zones.state',
  # :joins=>[ :us_county, :us_zone ],
       
  # :include rather than :join to prevent later 'select * from us_zones', etc... (which is stupid because the data's already in the resultset)
  # :include does an outer join when it should do an inner join, but can't find a way to configure that.
  # 1 outer join query is better than N*2 inner joins. (1 us_zones and 1 us_counties per location)
  # equally stupid... :include ignores my :select line and gets everything from all tables, including stuff I don't want like the giant us_zones.the_geom.
  # so which is less bad, way too many little queries or one way-too-bloated query?  I choose way-too-bloated for now.
  :include=>[ :us_county, :us_zone ],
  :order=>:user_label
)

http://www.slideshare.net/RowanHick/how-to-avoid-hanging-yourself-with-r...

A trouble ticket describing this issue is marked as 'wontfix'. Drag.
http://dev.rubyonrails.org/ticket/7147

Syndicate content