Find all the files that don’t have a specific line in them, or How I learned to stop worrying and love Linux

Monday, June 16th, 2008

So at my dayjob, we often have to do some funky things.  Like today, we are doing a refactor, and we wanted to know whether any of our web templates don’t use a component, because we needed to add something to it that would be global, and anything that didn’t use it would need to have our lines manually added.

With over 200 templates to look through this was looking like a pain to do, howver, in about 5 minutes, me and my pair managed to develop three solutions using just bash scripting.

My first was:

find . -iname \*.vm -exec bash -c "grep -Hiq javascript.vm {} || echo Not Found in {}" \;

While doing this, I was looking for the -q option on grep, to keep grep quiet when finding the files, and I found the -L option which does almost exactly this:

find . -iname \*.vm -exec grep -L javascript.vm {} \;

The -L says print the filename that didn’t match the expression on any of it’s lines.  (For the record, -l does the inverse, so finding all templates which contain a line is also easy).  Meanwhile Patric found a nice solution too,

find . -iname \*.vm -! -exec grep -q javascript.vm {} \; -print

This also relies on the return value of grep, but instead of some funky bash logic, uses find’s built in operators to execute the -print if the -exec returns 1

Did I say how much I like linux?

NFS exports and mountpoints

Monday, May 5th, 2008

I’ve had some weird behavior recently with my NFS exports on my server, and I think having figured out what was going on, I should blog about it, as it took me a while to work out why.

My server uses NFS to export some of it’s directories, most importantly is this line in the /etc/exports

/data    *(rw,async,no_root_squash,no_subtree_check)

I thought that this would export all my directories in /data, which it kind of does, but it doesn’t really do in the way I thought it would.

On my server, an ls in /data lets me know there are 4 directories, apps, downloads, mp3 and tv.  When I look on my client at /net/media/data I see 4 directories, apps, downloads, mp3 and tv.

But inside the downloads directory there are a whole bunch of files and directories on the server.  But on the client the downloads directory is empty.

Why would that be?  Why would a directory be populated on my server, but appear empty when browsing via nfs, it just doesn’t make sense.  But then I discovered this…

mib@media:~$ mount
/dev/mapper/data-home on /home type ext3 (rw)
/dev/mapper/data-mp3 on /data/mp3 type ext3 (rw)
/dev/mapper/data-download on /data/downloads type ext3 (rw)
securityfs on /sys/kernel/security type securityfs (rw)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
nfsd on /proc/fs/nfsd type nfsd (rw)

“What special about your mount paths?” I hear you cry

Well notice that /dev/mapper/data-download (an LVM volume) is mounted on /data/downloads, it looks like NFS only accesses the single filesystem that is mounted on teh directory accessed, and does not cross mount points that way the local filesystem does.

So /data is on the volume /, and has a downloads subdirectory, but that downloads subdirectory is a target for a mountpoint, and NFS does see the mounted data, only the target directory.

So how can I solve it?  The answer it turns out is simple, add the line /data/downloads to the exports file, execute exportfs -a, and restart autofs on my client (it caches) and it all started working.  I’m sure there is a perfectly sensible reason for NFS working that way, but I couldn’t find anything obivous in the docs warning me about it and I’ve been scratching my head for a week now trying to work it out.

I hope if anyone else has this problem that this post helps.

Evolution sucks!

Thursday, October 5th, 2006

In the linux arena, for those who dont know, there are two major desktop environments.

I happen to quite like both of them, I love the simplicity of Gnome, but like hte configurability of KDE. More importantly, I happen to really like the KDE applications.

I recently went from my KDE desktop to Gnome, for various reasons (I cant actually remember) and therefore switched most of my applications. Now for a lot of the apps, thats not a problem, Gaim vs Kopete - I hardly noticed, both can handle all of my IM accounts (see my contact me page), RSS reader, well I was using Liferea anyway, file browser etc, all fine.

But Gnome doesnt seem to have any kind of equivalent to Amarok, whcih is by far the best, most kick ass media player I’ve ever used. I keep my mp3’s on a server, and having a remote collection, getting it to update every day for any added podcasts, and play unlistened to mp3’s that are under the podcast directory. Easy to do and setup.

But the most irritating thing is Kontact vs Evolution. Now as an email application, there are some features of evolution that I like, I mean Kontact has a weird scheme that to move to next mail message you press the right arrow, not down, right! Taht weird right?

But trying to get my calendars to import into evolution is a pain. Kontact imported both my google calendar and the roadmap and task list from my trac repository without me having to really think about it.

Evolution wanted a URL that started webcal: to read my google calendar. I didn’t know waht it was, then after a quick google, found that it was just the http url to the ical file. Why start with webcal: and not http:? I wouldn’t have been so confused.

But Evolution wont import my trac ical file at all. the reason being that in order to connect to my trac repository you need a username and password. Evolution will read calendars over the web, but cant handle basic HTTP authentication!

After a number of minor irritants I think I’m finally going to move back to Kontact instead. Hopefully it works fine under gnome these days, without requiring a bazillion meg of memory to load up all the KDE libraries.

HOWTO: New Xorg+aiglx+compiz method - Ubuntu Forums

Wednesday, September 20th, 2006

HOWTO: New Xorg+aiglx+compiz method - Ubuntu Forums

Yes! I finally got compiz working on my dog slow integrated Intell 855 graphics chip on my laptop.

It doesn’t run brilliantly fast, but it does run, and now I have wobbly windows,  3d cube desktop and real transparency and whole bunch of toher cool looking effects.

I also reported my first real bug with Ubuntu linux.  The version of SDL that ships with ubuntu dapper doesn’t have a bug fix so any 32bit windows are transparent when  using compiz.  That includes anything using pyGame, which my game does, so it made it very hard to see what was going on.  Manaully installing latest version of SDL from website fixes it temporarily, and I’ve passed on the patch upstream, so there might even be updated versions of libSDL coming down sometime soon.

Instant Messaging

Tuesday, August 8th, 2006

I still cant get over how amny ways of contacting me there are now.

I worked it out, I have 6 differant email accounts, and a whole bunch of Instant messaging accounts.

Why aren’t more instant messengers interoperable? I know that the Jabber and XMPP stuff is trying to reach a consensus, and it looks like a lot of VOIP people are going to us ehte XMPP extensions, but getting older IM clients to change? I have a yahoo IM, MSN Messenger IM, IRQ IM, Jabber IM and Google Talk IM. In fact I have two MSN accounts and never sign into one, because I changed from my sirperc@yahoo.com account to my current mib@mibgames.co.uk account when I purchased the mibgames domain name.

Ah well, At least on Linux, I can install Kopete and I can have all of the contact lists for all of these and IRC in the same window.