IP address geolocation SQL database
December 1, 2009 – 3:57 pm
I’m sick of looking through chat logs to find an IP geo lookup database.
Now it’s here, possibly forever, or until I stop paying for hosting.
Setting up a remote git repository
November 23, 2009 – 11:49 amHere’s a great guide for setting up a new remote git repo. Thank you, Tool Man Tim.
PyDev Tutorial
November 20, 2009 – 1:54 pmI found a good tutorial on IBM on doing Python development with Eclipse. It might be a little out of date, but I think only the screens got moved around a little bit. It includes details on how to use ant, which I’ve recently started using with cruise control and PHP, so I’m becoming a fan.
I did run into an issue where I’d get the error ‘Variable references empty selection: ${project_loc}”, but a quick google brought me to a solution here.
I only had PDT installed, which doesn’t come with the Ant tools. You just need to install the Eclipse Java Development Tools. Found that tip on Stack Overflow.
Otherwise, a great tutorial on how to get working with Eclipse and Python.
Awesome Project: Sparkup
November 20, 2009 – 12:05 pmSparkup is a TextMate and Vim plugin for expanding short CSS style tags into HTML, written by my man Rico Sta Cruz.
From the github page:
You can write HTML in a CSS-like syntax, and have Sparkup handle the expansion to full HTML code. It is meant to help you write long HTML blocks in your text editor by letting you type less characters than needed.
Sparkup is written in Python, and requires Python 2.5 or newer (2.5 is preinstalled in Mac OS X Leopard). Sparkup also offers intregration into common text editors. Support for VIM and TextMate are currently included.
He even included a video demo:
Using Kickstart to Automate CentOS VM Creation
October 25, 2009 – 9:54 amI’ve done this using CentOS, but I’m pretty sure the same thing will work with RedHat the exact same way.
Virtualization is now a commodity with several free tools available from Sun, VMWare and Xen. If you’re like me, you like to create a new, clean VM for each experiment. However, this comes with a drawback – the installation process, choosing your timezone, putting in the same password every time, etc… takes a while.
Fortunately, CentOS provides the means to automate the process via a kickstart file. The best part is, after you do an install, you’ll have a kickstart file to start working with – it’ll save all your options in /root/anaconda-ks.cfg. You can edit this file later to include other packages that you’d like to install, such as the GCC c++ libraries. You can also run other commands through yum, and customize the system-wide bashrc.
I’m only going to go over how I did the install – for the full details check out the CentOS kickstart guide
I’ve created a minimal kickstart file to do a base install without gnome, printing, and any tools I don’t find useful. I’ve disabled X completely – I have no need to boot to a GUI. I’ve also uncommented the lines to configure the drive settings, so I won’t be prompted for anything along the way; my VM should be completely ready to go once I reboot after the installation. Pretty awesome.
There’s a convenient way to run a kickstart file off a http connection, so I’ve put it the web server on my dev box. My dev box’s IP address is 192.168.1.5, and I copied the modified /root/anaconda-ds.cnf (which I’ve renamed to ks.cfg) to the server. At the installation boot prompt, I’d enter:
linux ks=http://192.168.1.5/ks.cfg
Then let it run. After a bit it’ll ready to reboot and be used. Just make sure to unmount the install DVD.
Obviously this has a lot more uses than just creating VMs, but I’m not a sysadmin, so I haven’t personally used it outside the desktop environment. There’s other tools for different distros, and third party tools for CentOS/RedHat.
For what I’ve done so far, one could argue it’s easier just to create an image of the VM and to just clone that whenever I want a new one. That’s a good solution if I was interested in always creating the same exact VM. However, using the kickstart method has several advantages. I could create a simple web app that generates new configs based on what I’ve checked off in a few lists, allowing me to quickly fire up new VMs with the right software for whatever I’m working on. I can also upgrade the version of the OS and run the same kickstart, so I don’t have to maintain some up to date starting image.
If I’ve made any mistakes, please don’t hesitate to point them out. I’m by no means a guru on the subject, I’ve just wanted to look into this topic for a while.
Some Other Useful resources
VirtualBox – SSH’ing into a VM from an outside machine
October 24, 2009 – 1:43 pmNote: The terminology used below also applies to VMWare. The screens are different, but the issue and the solution are the same.
When creating a new VM through VirtualBox, you might have a problem SSH’ing into the box. You also might notice you get a 10.0.x.xx type address, even though the rest of your network is a 192.168.xx.xxx deal. Yes, the two are related.
Essentially what’s happening is the VM is sitting on it’s on little private line, where it can go out but nothing can reach it. By default, the network setting is “NAT”. If we want the VM to be accessible to the outside, we want to use Bridged networking.
This is easily solved, but requires that the VM be shut down. Go to the network second, pick an unused adapter (likely the 2nd one), enable it, and choose Bridged.
You should see this under the VM description now.
Go ahead and boot back into your VM. Fire up a terminal, run /sbin/ifconfig, and you’ll see you now have an ethernet adapter with an address you can reach.
Enjoy!
PyTables user’s guide
October 1, 2009 – 8:01 pmI finally found the manual for PyTables. In case anyone else is as blind as I am, I’m linking to it here.
Installing NumPy on MacOS X Snow Leopard
September 22, 2009 – 12:24 pmNumPy is a requirement to work with PyTables. This is the second step in the install process, after getting HDF5 set up.
These instructions are based on the ones found on the NumPy site, but I’m summarizing things for my own use later on.
First, you’ll need to install the Fortran compiler for OSX (gFortran). Fortunately there’s a Fortran universal binary installer.
Next, get the NumPy source. You can find it on Sourceforge.
Now, build with gFortran.
python setup.py build –fcompiler=gnu95
python setup.py install
Enjoy!
Issues Compiling HDf5 1.8.3 on MacOS X Snow Leopard
September 21, 2009 – 12:01 pmI’m trying to evaluate pytables as a replacement for very large Python dictionaries, but having some issues getting HDF5 installed on my Mac (OS X Snow Leopard).
I’ve been getting this error:
configure: error: C compiler cannot create executables
I haven’t been able to figure out what’s wrong yet – anyone have any ideas? I’ve got XCode Tools installed, I’ve compiled Apache, PHP and Memcached without issue (prior to Snow Leopard Update).
Help, anyone?
Edit: I just found a post in the Apple discussion boards about this being a Snow Leopard specific issue. The suggestion is to install XCode off the Snow Leopard disc. I’ve reinstalled XCode, and now it successfully runs the configure command.
Bad Web Developers Don’t Scale
September 11, 2009 – 4:30 pmOver the last 5 years, I’ve read so many articles about how X doesn’t scale. PHP, MySQL, SQL Server, Apache, you name it – everything gets a bad rap. Everyone has a different idea of scale and size, and sadly most people think their site with 1 million page views a month can’t handle the load because whatever technology they chose to use supposedly doesn’t scale.
Guess what folks – the problem probably isn’t the language you chose, or the database you’re using. Unless you’re actually big (many millions of pageviews per day), you can usually run just fine with a straight up LAMP stack on a few servers. The real issue is that your dev team has no idea how to write software or use the tools they have available.
At this point, I wonder – what is everyone’s expectation? At what point does someone say “oh, X scales great! One box is handling my web server, data storage, video conversion, at 20 million page views a day, all running on 1 nice little application!” Folks – pick the right tool for the job, and put someone qualified in charge of it. Don’t use MySQL if you really need in memory graph tree traversal (friends list).
Oh, and don’t even bother explaining to me the definition of scale. This isn’t an opportunity for you to point out what it really means – I get that you should be able to add hardware and get a proportional amount out extra output. Writing software that scales to a massive level is hard. That’s why there aren’t many companies that do it well.
/rant