Literary Twist Project and Run Basic

Standard

After tinkering with my Literary Twist Yahoo pipe (put in a book synopsis and it turns it into a synopsis for a horror novel) I’ve decided that it doesn’t work. Well, it sort of works when it finds any relevant words. The problem is, it relies on the common words appearing in the text fields that are entered into the synopsis text boxes and after testing it for a bit I’ve decided that this method isn’t good enough. Even though I methodically chose the words that would occur frequently enough, it seems that synopsis writers don’t like to write using common words ūüėČ

I’m going to try a different method now – Yes, I know that this project¬†has no practical use in the world, apart from¬†amusing myself, but¬†it’s a challenge to see if I can get it working in the way I wanted.

 

(c) Tomas Rotger (Flickr)

I’ve now worked out that what it needs to do is, basically¬†identify the most common words in any text that is entered into the text box (rather than common words in general) and twist or replace them in a way that makes sense, but also gives the horror aspect to the new words.

I realise I can’t do this with Yahoo pipes – it’s just too complicated to do it that way for my brain. I find Yahoo pipes is fine as long as I don’t make it really complicated and sometimes Yahoo pipes just stalls and sputters into lifelessness if I make anything too complicated.

So, I’m currently using Run Basic¬†to try and¬†achieve this. As the name suggests it’s a¬†language based around Basic – no sniggering! Basic is embedded in my brain and I will champion this favourite language of 80s school boys until you mock me so much that I curl up into a ball and cry. The good thing is that it’s server based, so you can¬†create dynamic web pages from it. I’ve used php¬†to create dynamic web pages in the past, but if I don’t use it for a while I forget the syntax/methods, etc.¬†Php¬†also tended to go wonky on me when I upgraded browsers as my programming was less than standard. Whereas, as I spent years programming in Basic, Run Basic was so easy to pick up. Run Basic also allows you to parse XML, manipulate files, and use HTTPGET, HTTPPUT functions, as well as other useful things.

So the first thing I’m doing on this new plan is to¬†put together a¬†term extractor and word count. It’s not quite there, but I’ve more confidence cracking it with Run Basic than anything else. I won’t let it beat me, no matter how useless the result is!

Literary Twist Update

Standard

I mentioned in my Tinkering Day post that I’d made some progress on the Literary Twist project. I thought it might be interesting for others to see what I’d done/how I’d done it.

Well, I’ve sort of done what I wanted on tweaking the words, but at the same time it’s obvious that what I wanted to do wasn’t enough to make it as entertaining as I wanted. ūüė¶

I basically got a list of commonly used words – I looked for a few sites that covered this to get an aggregated group of words… just to make sure I was replacing the best set of words. Then, using Google docs I pulled data from tables in websites into a spreadsheet, rather than retyping the info (Tony Hirst wrote a blog post about doing this). Sometimes, because the words weren’t in a table, I had to copy/paste the data into the spreadsheet. The data was a bit scrappy, as it came through to the spreadsheet in a variety of formats. Google spreadsheets doesn’t have a regex function and I didn’t want to do hundreds of manual find/replace, so I fed into Yahoo pipe to clean it up, using regex.

I output the clean file as csv and imported it into Excel, so I could get a count on the number of times specific words appeared. This helped me decide which words I’d do the find/replace on later on. I also needed to look at a few dictionary sites to make sure I replaced words that could only be used as one class of word, rather than more than one (ie adjective, noun, verb) – more than one messes up the syntax/form of the sentence.

Then I created a new Yahoo pipe, which had 2 text input boxes for title & synopsis. I added find/replace modules & manually entered words that needed to be replaced, along with text that replaced it.

Werewolf by Schnaars (Flickr)

Werewolf by Schnaars (Flickr)

Still, at this stage, some words didn’t work. Some of the replacement words didn’t work either. This is partly because I hadn’t thought too much about the type of text that would work with the replacement. For example, synopsis seem to talk more about ‘he’, ‘she’, ‘it’, rather than ‘I’, ‘me’ and this affects the way that you need to deal with the whole word replacement style.

I also worked out that, even though it’s a good idea to replace common words, because you’ve got a better chance of hitting words that can be replaced out of the 171,476 words in common use in the English language (according to the Oxford English Dictionary), more synopsis actually try to avoid the cliche/common words.

It still needs tweaking and it’s presentation still needs prettifying (or horrifying ;-)), but here’s the pipe for people to have a look at. All you need to do is enter a book title and synopsis into the boxes. I’d be interested in the output from anything you paste into the pipe, as I’d like to see how the pipe works on a wide variety of synopsis. Maybe anyone who uses it could cut/paste the output of the pipe as a comment to this post. I know it needs work on.

Tinkering Day 1

Standard

As I couldn’t go to Dev8D and I probably won’t get to go to the next library mash event ūüė¶ I thought I’d have my own day of tinkering. I probably gave myself a bit of a long list of things to try, but I managed a fair bit.

(1) Made a bit of progress on the Literary Twist project – Scraped a list of commonly used words, worked out phrases to replace them and put them into all into a Yahoo Pipe. Need to tinker with it though, to make sure it comes up with consistent and accurate output all the time, no matter what user puts in. I think I could do a bit more with regex too, but that might be another stage on. Also learnt trick for avoiding problem of pipe not running properly. Also realised on another pipe that Yahoo pipes can be overloaded and might not be great for lots of feeds going into one pipe.

(2) I also tried to work out if you can pass query to flash player. You can, but not with the one I wanted to do it to – might try to find a paste and run script, if such a thing exists.

(3) Basic programming – My first programming language was Basic and I still love it. I know some programmers look down their nose at it, but I don’t care. It’s easy for me to remember. I think my head is still in BASIC mode and that’s why I’ve struggled to suss out Java. I’ve made progress on Java, but it’s taken me a bit of time. Anyway, I got a yearning to tinker with my old MSX (still in cupboard) and decided to download an MSX emulator. I could still program in BASIC on it after so many years of not bothering, but I couldn’t save files on the hard drive. I wiped it of my computer in frustration and looked for a browser based basic instead – just out of curiosity.

I found runbasic ūüôā , which looks straight-forward, but will allow me to develop interactive websites in a language I’m familiar with. So, I’ll be tinkering with that, even though I’ll still be using Java too.

(4) I also tried to get to grips with Gamemaker. I’ve had a bit of success in putting very very simple games together on it in the past. However, even though you don’t need to know how to program to use it, I think if you do know how to program you might as well program, as it’ll be just as quick. It’s not as straightforward as pointing and clicking. So, I think my tinkering time will be better spent somewhere else.

(5) I also had a scout around for free Java and php hosting – just as a way to test out some programming on a live site. I found a couple of sites via Free-web-hosting. They don’t get massively high scores, but I just need somewhere to tinker on a live system, rather than on my home test system (Apache server).

I’m quite pleased with what I achieved in my first full day of tinkering – sorting out some of ‘Literary Twist’, flash player queries and tinkering with runbasic. Even where I didn’t make any progress (MSX emulator, Gamemaker) it meant I could wipe them off my list of things I want to get to grips with and move on.

Literary Twist Project

Standard

A couple of months ago I came up with a daft idea based on¬†reading ‘Pride and Prejudice and Zombies’ (a novel that has since spawned a new genre of classic/horror crossovers). I was really disappointed by the book and smugly thought ‘I could do that!’

It just seemed like the author had chucked in a few ‘unmentionables’ and a bit of martial arts and released the literary monster onto¬†the world. So, in a poor attempt to take the mickey, I set up a Yahoo pipe that would allow people to paste in a title and description and the pipe would do a simple find/replace action and turn the text into a cheesy piece of Hammer horror.

Zombie Librarian, courtesy 'wvs' on Flickr.

It worked on a simple¬†level, but I thought I must be able to improve on it. I haven’t really tinkered with the idea since then, but have felt it’s probably worth pursuing just for fun & as a personal learning exercise.

I’ve decided to call the project ‘Literary Twist’.

There’s a number of stages I’ll need to go through to get something decent up and running.

(1) Work out a set of words used frequently enough in the English language that would appear a high percentage of the time in any text a user entered into the description, or use term extraction to pull out enough relevant terms in the description that could be replaced.
(2) Decide on appropriate words that could be used to replace the original words and feed them into the replace procedure.
(3) Possibly if someone entered an ISBN, author / title I’d be able to pull in a synopsis from a book catalogue instead of using manual text entry?
(4) Do the process of find/replace
(5) Output the result as a decent looking web page (all nice and purdy)
(6) I’d like to have a nice user-friendly¬†interface (a form) to the pipe as well, which I know I can do via html and javascript, but I’ve never tried.
(7) I’d also like to go beyond the horror genre and allow users to select the genre they want it twisted into (eg romance, western) and I’d want to do it all in the same form/screen.

So, I think I’ll give it a go. I’m not sure how long it’ll take, but I’ll be keeping track of it here.