Tag: programming

Looking at Apache server logs

Posted on by Liz

I poked around with Apache log files today, looking at the log file formats and thinking of ways to look at the data. While a nice log parsing tool like AWStat or maybe a slick one like Mint would be nice, I just wanted to look at the data without too much fuss. Also, it’s not my server. So I wrote something to do what I wanted with grep and sort and uniq. A bit afterwards, I realized using awk would give me the fields I wanted from the file, and would be easier & faster than using grep or egrep.

So here’s some useful links,

* Managing and parsing your Apache logs This is good and simple with some useful example scripts.

* Checking your system logs with awk This has some explanations of why awk is handier to use than grep for handling log files.

* I’m growing more and more fond of awk! Why didn’t I know this before – I did everything with Perl, instead! *mad crush on awk*

It was fun doing this and made me want to install Apache and get it running so that I would know its ins and outs a bit better. But that will just have to wait until I finish digging into these log files… It is amazing what you can tell from them and what you can deduce, given a huge web site to sink your teeth into. Right now I’m thinking about bots and spiders, remembering the fun things I used to do with spam hunting, filtering, and watching the habits of users going through a big proxy server.



Liveblogging take two: BlogHer Boston

Posted on by Liz

Our workshop was for Blogging Basics: personalize, polish, and promote your blog. Danielle Henderson, Alissa Kriteman, and Meghan Garnum joined me on the stage. Kristy got us in touch with Elliot, the hotel employee setting up the projector, who found all the stuff and the cables and hooked everything up. Also, NeoOffice worked like a charm! Around 100 people filled up the room. We took some goofy photos and were having fun chatting before the talk. Everyone in the room was so talkative and intense with their conversations that I had to say “SEX” into the microphone several times to get them to pay attention. After a brief intro and my pep talk about getting your hands dirty digging into your code, we each talked about our experience. Alissa, Danielle, and Megan were full of enthusiasm and confidence, knew their technical stuff, and were great speakers.

Alissa Kriteman is the host for Just for Women: Dating Relationships and Sex. She has a weekly audio podcast show about empowering women. Alissa has been blogging for about two years and is enthusiastic about her future in blogging on WordPress. Hey, Alissa is from SF! I could hang out with her and hack on blog stuff! Yay!

Danielle Henderson has been blogging for 6 years with a variety of hosting platforms. (Bio: Danielle Henderson is a 31-year old college sophomore and freelance writer. She’s been a blogger for close to 6 years at Knotty Yarn, and still has no idea how or if that is applicable to common conversation. Her first book was published in 2004, and she was recently featured in Cringe: Teenage Diaries, Journals, Notes, Letters, Poems and Abandoned Rock Operas.)

Megan Garnhum has been blogging for nine years and has over ten years experience in marketing companies, media, software and online social networking. She also does blog design and consulting. (Bio: It was love at first blog for Megan, who began blogging back in 1999 and hasn’t stopped for a breath since. A hobby at first, blogging has become a big part of Megan’s life. She has taught herself how to create, build and manage blogs on all of the major blog platforms and now designs and provides blog and web consulting for individuals and businesses, through Webundance, her consulting company. )

I’m Liz Henry and am a producer and software developer at BlogHer. In addition to being a poet and literary translator, I’ve been writing online since before 1990 and blogging seriously since 2003. When I started blogging I didn’t think anyone would ever read it. I was so wrong. I blog about my life and about many topics including feminist science fiction, technology, and social media.

We asked the audience who they were.

More than 2 years blogging? (About a third)
More than 1 blog? (Half the room)
Blogger or blogspot? (Most of the room)
Typepad? (5 or 6)
WordPress? (a third of the room)
Tumblr: 1
Twitter, Plurk: 10
LiveJournal: 2
Drupal: 2
MySpace: 2

We discussed those difference a little and dedicated to break groups by experience level.

People who ever have messed with their code beyond a minimum. 1/3 of the room.

I mentioned my experience doing tech support for the BlogHer ad network bloggers. Most of the requests for help with code, the bloggers put themselves down, call themselves idiots or dolts and explain how they don’t know anything. It’s okay, but it makes me a little sad and I want to help people learn the skills to be more confident, and approach computers and approach things they don’t know with a different attitude, not feel bad about themselves. Try not to do automatic disclaimers or self-deprecation. Everyone needs debugging help. I do too. I am often able to walk people through difficulties, but sometimes I just get passwords and fix problems. So if I can persuade you all to feel confident about playing with your blog code I’m making my own job easier. *audience laughter*

Also, the more intimate you are with your blog…

Alissa: Ooo baby, intimate!

… the more intimate you are with your blog, the more closely you can make it be how you want, for self-expression or to better reflect your business purposes.

Pep talk. Examples of some virginal blogs that have never know the hand of woman. The default blogger.com blog. The default wordpress blog.

What would this “generic” blog template say about you to your readers and the rest of the world?

Here’s a myspace blog, very blinged up. What people think is good design doesn’t matter. What is your goal? Clarity, or obfuscation? This is good blog design, conveying a strong identity:

And clearly, this blogger dove fearlessly into hacking her blog’s back end, to express herself.

Look at your code. Engage with it. Print it out, put it next to the pages it’s making. Mark it up and draw on it. Figure out which bits are doing what.
Confusing code is annoying. (Slide of WTF CAT)

It is a good thing my slides are working because otherwise all of us up here were going to act out the LOLCATS. (Alissa and the others make the WTF cat face. We’re a funny crew up here!)

Pastebin.com is very nice. Paste in your code, label what kind it is – html, php, etc. And it will give you a temporary url showing all that code nicely highlighted in color. You can send that url to someone over IM and get debugging advice. This is amazingly useful. Don’t be embarrassed to ask someone else to look at your code and spot obvious mistakes. It’s just like copyediting.

Indenting properly will make your code more readable, too.

It will start to make sense. You don’t need to know everything, to hear blah blah blah STYLE=”width 160″ blah blah blah. (blah blah Ginger) Learn bits at a time, just the bits you need to fix.

Danielle: You should look at the documentation. WordPress, Typepad, Blogger help files are good.

Megan: Yes. Search within the typepad web site. Very helpful. It’s how I learned everything.

Me: We’re all self-taught; that’s very interesting.

Back to my talk. View source on other people’s blogs and find examples of things you’d like to do, cut and paste their code, copy and tweak it, i
t’s how programmers learn stuff. Backup first, then experiment. Ride it like you stole it!

Onwards to the “Personalize, polish, and promote” part of the afternoon.

Who are you? Reflect it. Whether your identity is actually you with real name, or pseudonym. Contact info. About me sidebar box or page, or both. This is built in to most platforms. In Typepad you just check a box and fill out a form. Same in Blogger. In WordPress you can make a page for it. (Slide with examples of About Me info. Some are short, serious, funny, some long and detailed with every interest of the blogger listed.) Include links to, or content from, other places you “are” on the web.

Who are your readers? Reflect them too! Expose their identities to each other. Readers on a blog are aware of each other, sometimes become a community. Allow readers to link back to their own web presences, email addresses. They can have profiles on your blog. Avatars, icons, gravatars so they have a photo too in their comments.

Why do we care?
This is important not just for touchy feely reasons or SEO.
It is important in the history of literature and ideas. Women’s diaries, letters. Intertextuality. Intellectual crosscurrents and crossfertilization. Your future biographer might like to know who your blogfriends were in 2004 vs. 2008. Or that you regularly read and commented on Pharyngula or Feministe — or on BlogHer. Consider yourself as a writer and as part of history. Represent your connections. It is important in feminist or in women’s history. (Slide with books of letters by Virginia Woolf to Vita Sackville-West, and letters of Francis Ellen Watkins Harper to friends and newspapers, and book covers like “800 years of Women’s Letters”. ) We are in a mass movement where women are writing in a public forum, being part of public discourse. It is new in the history of the world.

What are you writing?

You might set out to write on a particular topic. then shift over time. The life of a blog. identity, subjects change. Tag your stuff. Categorize it. Make your post titles relevant (if you care about that, and want things to be found. If you don’t want it found by a casual searcher or reader, bury it. Top posting buries. Baroque digression buries. Multiple subjects in one post buries the content.

Self-discovery process to tag your content. I went through one of my blogs and tagged, maybe 300 posts. This took me a couple of evenings. It revealed what I was actually blogging vs. what I thought I was blogging. I thought I was writing about poetry and translation, and I was really blogging about politics, feminism, the internet, and social media. WordPress has a very nice plugin called Simple Tags that lets you machine-tag all your entries at once. You can batch tag in detail, then combine tags to get higher level categories.

Tag, tag, tag. Tag clouds! Delicious (murmur of agreement from audience.)

Megan: Make your links relevant. When you make a link, don’t put it around the words “click here” or “link”. You want to put it around “really great shoes”, or whatever- some text that is related to the meaning of the link and the content of the page you’re linking to.

Me: Your future biographers will thank you here, too. The text you choose to hyperlink is probably important so in any textual analysis it would be weighted as more important. Your future biographers don’t want to know that you’re blogging quite a lot about “click here”. They want to know you blog all the time about really great shoes. Anyway, any meta information about your content is great!

How to do this stuff?

1) Look through all your blog’s settings and options. There may be built-in “About Me” or Recent Comments options.

2) Javascript widgets!
* Copy or download scripts from other people.

Blogger: Widgets, Gadgets
* Search for interesting widgets or code snippets
* Copy and paste into the “Javascript/HTML” box.

WordPress: Plugins
* Download/upload to your server
* Unzip the files
* Activate them in the wp-admin panel

Typepad: Typelists (basic templates)
* Create a “Notes” Typelist. paste in code
* Select content – check off a box to include
* Order content – drag and drop sidebar widgets

Hack with your friends!!! Thinking about gender, it is my observation that men tend to grow up doing projects together. Guys in middle school flail around in linux or whatever, programming together, learning stuff, not embarrassed to know nothing, they have no shame. Girls, not so much. In my little feminist techological utopia we would all be hanging out programming together and learning stuff. You don’t need an expert to learn from. Peer mentoring, just having someone to look at stuff with, it is good for moral support. Another pair of eyes for debugging is great. Get together with a blog friend and work on your blogs together over coffee.

Questions and statements from the audience. Why is everyone switching to WordPress blogs? Are they better? Is that just a trend? How to compare platforms? Panelists explained that Typepad cost money, Blogger was free but slightly less flexible, WordPress more setup but much more flexibility, Moveable Type good but have to get into the code more. You can search “compare blogging platforms” and see what comes up for an explanation. As consultants (or just friends) we tend to advise beginning bloggers who don’t want to do a lot of setup, to use Blogger or Typepad, or hosted WordPress. Then migrate up only if you need more features than what you get for free. Or, try all the platforms (it is easy to set them up) and see which you like best. Look at a blog you like and see where it’s hosted.

Statement from audience (I think this was Christy Matte): What Liz said about putting your own email address on your blog, in the clear, she disagrees nand thinks we should mention contact forms. You can set this up on any platform. Plugins or a third party service. Or you can obfuscate your email address to try and confuse people who harvest your email and sell to spammers. I commented that obfuscation isn’t really going to stop screen scraping and email harvesting and it’s better to fight spam with filtering in your email reader. I think Danielle or Alissa recommended using gmail as its spam filtering is quite good. Captcha and recaptcha were also mentioned for comments and so on.

Question from audience: How do I find someone to just redesign my blog and make it nice? How much should I expect to pay? How do I know if they’re any good? Answers from panelists. Megan: Well, I do that for my business, Webundance, so you can ask me! Megan’s rates seem very reasonable for basic work. Me: If someone is trying to charge you a thousand, two thousand dollars, you’re being ripped off. Alissa: Students or kids might help you for very cheap. Me: Look at someone whose blog design you like, and ask them who did it. Ask the designer or web developer for client referrals and examples of their work and ask their clients if they’re happy.

Question from audience: What the heck is the difference between tags and categories? And how can I get my blog search widget to stop searching my sidebar? Because it detects the tag cloud and category list for every single post, so it thinks all 500 posts on my blog are about pumpkin recipes. Answer: Ugh. We are not quite sure. The
re must be a better search widget for your blog. We’d need to look in more detail what you’re using and how it’s configured. Try the Geek Lab?

Our last half hour was spent in breakout sessions. We split up the room by level of experience. Alissa went to the back of the room with newbies. People who considered themselves fairly experienced in getting into their code were at the front.

Other notes:

Hacked with leadingfromtheheart.org a long time on her wordpress recent posts plugin. We modified the plugin code that she’d already modified. We broke it, she re-installed it, then we ignored the plugin and went for fixing the styles of the stuff that the plugin spits out: 

 li, h3, ul, and a

. The mysterious space before the recent posts turned out to be a top margin on 

h3

that was 3em, not 3px. Whoops! I showed her how, if you view source on someone else’s blog, you can search on “css” and find the link to their style sheet, and then paste it into the address bar to see their whole style sheet in the browser. So, for example, I used my spying skills to find her stylesheet: http://leadingfromtheheart.org/wp-content/themes/unstandard/style.css . Anyway, she’s a good hacker and has an amazing, amazing blog about teaching high school. Give it a read.

Talked with Consuming Lilly who is super-de-awesome, about her blog, about the history of food and feminism and cookbooks throughout history, and we looked at her blog coding issue: she has customized her blogger.com template to the max and lost her way a bit. We looked at it for a bit and I ended up telling her to print it all out and mark up each section so she would know exactly what bit of code controlled which page element. Recommended Firebug and Web Developer Firefox plugins. (Not for the last time!) A bunch of us were gathered around looking at her blog template. Did I mention that it’s pretty!? It is!

Someone, I think Lilly, gave a good warning. Look at your code, if you get a template from someone’s web site, because template-makers slip Google adsense code in there that pays out to them. On the one hand, you got a free template from them. On the other, it is unethical not to be upfront about it if you’re doing that in your template. Be warned!

Talked with: chestercountymoms, bestisyetunwritten (debugging, ad network questions)

There were more people I helped out for a minute or two, or that I got talking with each other.

Long talk with Megan’s Minute about design and usability. Hey! I heard about people liking her handbag of the month thing. (Though I don’t use handbags.) She asked me for advice on what to put in the sidebar, and where? She hacks her Typepad sidebar a lot. I said to put in polls, most popular, content of the recent comment not just person + post title; put TV show reviews up high because they are a particular focus (Lost, Survivor, etc) and her readers love them; shorten headlines that are long and go across more than one line.

Megin from Chester County Moms wants to put an rss feed of all her blog contributors in her WordPress sidebar. We looked at the built-in widget (design tab, widgets, click “add” to add it to sidebar, then edit to add the feeds.) She wants a fancier feed with some randomization because she has 25 contributors! Annette Krasow suggests some aggregation tools.

Megin told me she got her header graphic from a photo, and then — I missed it in the confusion — there is some iphoto plugin that builds caricatures? Or was she saying she paid someone to do the caricatures from a photo?

Talked with Christy from about.com family computing. She’s really fun and intense and I wish we got to talk more!

Talked with Megan Taby from Freepress.net and savetheinternet, about feminism and net neutrality. I look forward to talking with her more!

I left the workshop very high off of it, feeling like everyone got something useful from it. Also, I was glad my slides worked and my lolcats got a laugh.

Talked with ecochick.ca about her javascript and css issues, very convoluted, not possible to solve, depended on a 3rd party. I gave some advice on how to frame the problem to their support folks, and recommended for some other layout issues that she get the web developer Firefox plugin.

Talked with Megan Garnum some more. We were laughing about being very social and outgoing but then needing to get with our computers and shut up. “Could I just crawl inside you, laptop?” she said. I agree. We were both crashing and burning and needed to shut up and blog. So, of course, we talked about that. I think we might have also twittered it. Not really…

Looked at Sexpertise, Isis Inc. Talked about Planned Parenthood use of mobile apps and texting. Sarah and I sent a question to ChaCha: “My condom broke. What should I do?” and they sent us a dangerously stupid answer: to put vitamin c tablets in our vaginas. What horrible misinformation. I will be writing to ChaCha about it.

Looked at nakedanarchist blog. A kindred spirit, I think!

Gossip with Sarah Dopp about Geek Lab and what people’s blogging questions or issues were. Lots of rss feed questions: how to aggregate and post feeds on blog sidebars, how to set up a feed reader, how to put rss feeds on your blog (answer: they are already there, probably) and how to tell what your feed address is. How to put plugins in sidebars at all, in various platforms. I also got that question a lot, but more often I got very specific questions about css or “what plugin would do X thing for me.”

– Keynote panel by Elisa Camahort. I got too tired to take notes.

Cocktail party! I sat with a fun table of bloggers and gave them lots of stickers from my secret sticker collection! Tracy from leadingfromtheheart, Liz Davis, Sherry Pardy who has twins with the same name as Tracy’s sister’s twins, almostfoodies, Melissa from thecollectionspace whose husband writes about music. She told us a great story about the Tape Guy who makes sculptures of people out of scotch tape and who her (mystery industry discreetly not identified) conference was going to hire to be sort of a quirky entertainment to perk people up, but unfortunately the Tape Guy broke his hand and couldn’t do it. We all exchanged cards. Someone (almost foodies) reacted in mock horror that she might be accidentally hanging out with *mommybloggers*. Really, you can hardly tell who we are when we’re drinking cosmopolitans and in stealth mode. THEN WE BUST OUT THE WALLET PHOTOS. Muahaha!

I’d like to say that all my co-pilots, or co-panelists, were amazing, smart, great speakers, very pro-active about grabbing that microphone. Everything they said was a great contribution. In fact, everyone from the audience who raised their hands and spoke up had great points. There was some lively discussion of whether to put your email address in the clear on your blog, or not, with strong audience and panelist support for using contact forms. (While I am in favor of just putting your email addy out there.) Other people made great points and shared relevant information!

Am now decompressing. Sarah and I are laying around blogging and trying to be as antisocial as possible. I consider declaring a pajama party with my room number on Twitter. Would that be a mistake? Am I too tired? Should I stop typing? Someone hose me down.



Greasemonkey and jquery tricks

Posted on by Liz

This took a while to figure out and then I forgot it all again. I have some Greasemonkey scripts that fiddle around with menus and automatically log me in to a difficult-to-navigate HellishSite. They use jquery and I thought I’d record what to do and pass on the tip.

When I used jquery the old way in my scripts, it broke HellishSite’s layout probably because they use jquery in some way themselves. Something about the insane way they use frames might also be the culprit. I didn’t look too deep into that. Here’s what works. Use a @require line as described in the Metadata block docs for Greasemonkey:

// ==UserScript==
// @name          Hello jQuery
// @namespace     http://wiki.greasepot.net/examples
// @description   jQuery test script
// @include       *
// @require   http://ajax.googleapis.com/ajax/libs/jquery/1.2.6/jquery.js
// ==/UserScript==

$(document).ready(function() {
 $("a").click(function() {
  alert("Hello world!");
 });
});

The other trick, the bit that was driving me completely crazy, was that in order to get your @require statement to work you have to uninstall the script and install it again. Otherwise, the browser won’t read the new info in the metadata block. To do this, you go into the “Manage scripts” control panel for Greasemonkey, edit the script and make sure you copy it somewhere accessible, highlight the script and check “uninstall”. Unfortunately there is no “reinstall” button. In your browser address window (on a Mac) point to your copy of the folder you saved it in like this: file:///folder/ and in the browser screen, click on the saved javascript file. It will magically ask you if you want to install it. Yay! Install it and this time it will work. Your other option here is to go edit your Config.xml file where your browser is actually looking to get the metadata when it goes to execute the script.

In the script itself I am doing just a few things.

Testing what page I’m looking at, just by seeing if a particular field name is on that page:

if
($("input[name^='lastName']"))
{
(do some stuff)
}

Filling out fields:


 $("input[name='loginId']").val("Sucky");
 $("input[name='password']").val("blowme");

(the non-jquery way to do this is something like, 

document.getElementsByName('password')[0].value = "blowme";

Ugh, shudder…)

Selecting from dropdown menus:


 $("select[name='profile']").val("sucktastic");

Pre-selecting a bajillion clicky things from horrible little fields full of options:


 $("option[value='8008135']").attr("selected","Selected");

Making horrible little fields bigger, that show 4 options at a time out of possible 25:

 
$("select[name='section']").attr("size",25);

All this will become unnecessary once I dive into the HellishSite API but right now I don’t have time to go that route.

A note that I wrote this stuff back in March, adn then it broke, I fixed it, it broke again, I had forgotten it all and had to re-learn it, then my hard drive crashed and I lost my fixes and have spent a while fixing it again. This is pretty typical for me. I learn enough to make what I need, but I don’t go out and become a Greasemonkey or javascript expert. Just enough to get the job done and move on.



How to create a web form to talk with Google Docs

Posted on by Liz

My friend and co-worker Jenny from Three Kid Circus pointed out a cool feature of Google Docs to me today. I use Google Docs spreadsheets mostly as wiki-like documents for a group of people to edit a web page in table form. In other words I use them for layout of a web page, and maybe for sorting columns, and sharing fairly simple information.

In another conversation I had recently, Elisa Camahort was talking about wanting a simple widget sort of thing for party RSVPs, but that would be more than a link to her email address – it would be a form for people invited to fill out. So, it occurred to me that the form-constructing capability in Google Docs would do all the things she wanted.

As I thought further, I realized it also might be helpful to my friend Squid Rosenberg, who has a son whose educational plan is managed by a fairly large group of people, using Google Docs, tracking his progress and patterns with a daily record. Similar documents could be very useful for anyone tracking health care issues, for a team of caregivers for an elderly person, for example.

Since this is a fairly new and obscure feature, and somewhat of a convoluted process to click around to where you even understand what the feature does, I thought I’d describe how to set it up.

1) Go to Google Docs.

2) Create a spreadsheet and call it “My potluck dinner” or whatever.

3) Make columns for whatever you like — name, email, # of people you’re bringing, what kind of food you will bring to the potluck.

4) Click “share”.

5) Click “invite people to fill out a form”.

6) Click “Start editing your form”.

7) Click the Save button.

8) Click “Choose recipients”. (You don’t have to put in any actual recipients.)

9) Click “Embed”, in the upper left corner.

10) Copy the code and stick it in your blog

Each column in the spreadsheet shows up as a title and text input field in the form! You can also add new fields.

It looks like this:

From there you might have to twiddle the formatting to make it look halfway decent. It is not set up to look nice in a blog sidebar, but it looks just fine in posts or pages in TypePad, and Blogger. It doesn’t work in LiveJournal or WordPress because it uses iframes, though in WordPress you can install a widget to get iframes to work.

The resulting input into the spreadsheet looks like this:


(click image to see it bigger)

Note the timestamp, automatically added!

So, this could be useful for many purposes. My potluck dinner example is frivolous, but anyone who’s suffered through the annoying interface of the spreadsheets on Google Docs, or who uses them for individual or group data entry, could set up extremely easy web forms with this hidden-away feature.

I would like to send a polite nudge to the Docs development team to uncover the forms feature! It would be very nice if it were an option to click directly from the spreadsheet or other document.



Code that isn't at all poetry, but that is structure & patterns

Posted on by Liz

Happy Poetry Month! Rather than poeting, for the past few days I’ve been twiddling with code. It is much the same state of mind as translating, or basic composition, but for me at least, not quite poetry. It does require moving a bunch of words around, arranging them, and imagining their interpretation, organizing words in order to have an effect. For poetry or composition, an effect on a listener/reader, so you are imagining a logical and emotional state and the interpretation and effect of a person. For code obviously you are writing so that a machine will follow your orders perfectly; but less obviously you are writing for yourself in the future and for other future, human readers of your procedural pattern of thought. You are writing for your future (self or other) human, so they can modify and extend that code and put it to other uses. In other words, it has a bit in common with an oral or folk tradition. Repetition and patterns are good in poetry if you want to create structure for extension and improvisation.

So, just now I was doing some of my baby-Perl for some contract work. And the deal is, there are a bunch of users, and their accounts go through various bureaucratic steps, and through various work people and departments, some steps requiring others to happen first, for the account to become fully active. This is a fairly common situation for any institution. So, I had a Perl script that would take some command line options and then would do various things with the user and account data. As more people started realizing I could manipulate account stuff, and could generate reports, etc, they started asking me for new tasks. So, the hacky little script grew very quickly to a giant horrible tangly mess full of regular expressions that I did not understand anymore, 30 minutes after I wrote and tested them. A reg exp is a thing of beauty but it is not a joy forever. Instead, it makes my head hurt.

So I started about 4 times over this last month to rewrite that mess to make it easily understandable and extensible. I scribbled and thought on post-it notes so I could try to break down what needed to happen into chunks that I could move around & visualize, easier than in a 200 line text file.

It went kind of like this: use GetOpt::Long to tell from the command line what kind of report or change is required. Log in to several systems. Iterate over a range of account IDs in a big loop. Then do some http page getting and parsing. Then a lot of if else statements to see what command line option is turned on. Mixed in with some more tests and if elsing. Then again depending on command line option, do some other junk, write to some other web pages and outfiles. At the end of all that looping, write some more outfiles.

Ugh! You can see that any new capability meant that I had to do more page parsing and more reg exping, as well as thinking through all the logic of the whole if-else mess.

Today I suddenly realized several things. Speed doesn’t matter for this. I can set it going and let it chug away.

So, number one, for each user ID, just read in all the possible pages that have info on that account. It is only 5 web pages on 3 different systems. Read them in and parse out all their fields.

Number two, think of each account as having a state. There are 8 different bits of information that change account state, out of the 50+ possible bits of info. So, after parsing all the pages, look at the 8 pieces of information I care about, to determine the account’s state.

For things I then want to do, they fall into two categories. Reporting and state change. Reporting is easy. For changing account state, I can define for each case of one state to another; what actions it takes to change the account state. There are objects, and states, and transactions.

I have never really understood object-orientedness no matter how many times I think about it, and use and write code that is in theory, sort of object-oriented. It’s not like I get it now, but I get it more than I did.

Suddenly everything clicked into place and I understood how to write the code in a way that would be useful and elegant. I understood the root of the problem. It all fell together in a system. It looked like a pattern, like information that was beautiful. I know, it is a bunch of account data in a bureaucratic procedure. After years of being “programmer analyst” doing back end tools for university departments, I had to find beauty where I could. The “click” feeling means I look back on my month of sporadic attempts to write this program, and it looks like I was brain-deadishly trying to make something out of legos by gluing their corners together, when all the while I could have been snapping them together how they are supposed to go. But, before, I could not see the intersections.

So, just now I had the exhilarating (yet slightly shame-faced) feeling that I had just reinvented the wheel, or some basic principle of computer science that if I had any sense, I would have known from taking some classes. On the other hand, taking computer science classes doesn’t guarantee you know what you are doing or can build something that other people find useful & usable.



the poetics of programming

Posted on by Liz
I’ve been getting tech recruiter calls steadily for the last few years. They’ve stepped up in the last couple of months. The deal is, I don’t have control over an old server I used to be on, and so can’t change or take down a set of old web pages including my resume from 2001! So headhunters see 1999-present as a programmer and start drooling. Since I lost my job in late 2001 in the crash, and had 6 months of futile jobhunting… I have not kept my hand in with the programming. I told this to today’s recruiter, who blustered, “Aw, you’d be back up to speed in a month!” While this is true, I’m not sure where I stand on going back into a tech job.

If the people were super smart and nice… and know how to communicate… and are sane… and the job was poking around in someone else’s giant mess of Perl hackery and twiddling it… I would likely be quite happy. There is something nice about a huge data set and messing around with it and “seeing” into it various ways. On the back end – not live. And about the process of understanding someone else’s code… very much like the logic of translation. I especially enjoyed the spamhunting part of my old job, where I could imagine being a keen detective or a spy. I’d go back to translating 1 night a week and on one weekend, like I did when I worked at That One Dead Search Engine Company. But is the company high-pressure and will they want me 12 hours a day? No way could I do that!

The thought of having oodles of money again is tempting. I could save it up like crazy. And yet I could also be quite happy with my plan of teaching community college part time and finishing the Wittig book and then the huge anthology.

going to talk to them. Mostly because their site is good and slick. If it was dumb I would not be tempted for a second.

I’m on the fence!