I have recently been very interested in what parser IF (that’s text-based “interactive fiction”) might look like in the “post-PC” world brought of mobile computing. What do we do with a game whose core interface is typing, when we’re operating on devices where keyboards, while available, are less convenient and much less central to interaction than on desktop machines? Given the fact that the form factors of smartphones and tablets are encouraging users to explore new forms of interacting with text–as well as to rediscover pretty much unreconstructed older ones–now is an ideal time to consider how we might refashion the interface for parser IF.
[Updating the IF interface more generally has in fact been a topic of interest in recent years, even without any particular reference to mobile computing: The experience of most users has drifted very far from the command-line interface where most IF remains mired, and there has been a recognition that expanding interest in the form will very likely require changes to the interface. For some of the most salient work, see here, here, and here.]
The present experiment is a simple conversion of Sand-Dancer, Aaron Reed and Alexei Othenin-Girard’s example game from Reed’s excellent book Creating Interactive Fiction with Inform 7–thanks to Aaron for his generous permission to use it here! As such, it is a full work of parser IF. However, the player can use links and buttons to play the game through without typing anything at all. (Actually, I haven’t played the experiment all the way through to see whether I’ve accounted for every section of the game. It’s quite possible that I haven’t–but in principle the interface would allow most standard games to be played all the way through without resorting to the keyboard.) You can play the experimental demo here, using the Quixe interpreter. It will look best on a WebKit browser (e.g., Chrome or Safari) and god knows what it will look like on Explorer. Unfortunately, the browser interpreter is a bit slow; perhaps ironically, it’s too slow to play on a mobile device!
Screenshot of an early moment in the experimental demo.
The window is divided into multiple panes, each presenting a different type of information. In the main window, where the story’s text unfolds, the only interactive elements are hyperlinked objects. Similarly, the player’s inventory is shown off to the lower right, where the linked objects are always available. Click on any link in either of these windows, and the selected object will be highlighted in the upper right window, where its description will be followed by a series of buttons that describe actions that can be done with, or are related to, the object. At the bottom of the screen is another set of buttons representing the player’s navigation options, and above these is a narrow area where commands can be typed if the player wants or needs to enter free-form text. Buttons and links alike are implemented as Glulx hyperlinks–they are merely styled differently using CSS with the web interpreter.
For the most part, the entire overlay is procedurally generated. Aaron used the “Get BENT” system that he discusses in Creating Interactive Fiction with Inform 7 throughout Sand-Dancer, and that means that it is easy to use Inform’s “rule for printing the name of” activity to wrap the names of all important objects in links. After a link is clicked, we use object properties (along with a few special-case items) to generate the buttons in the object window (e.g., if the object is a closed container, provide the choice to open it). We also use the built-in conversation suggestions to generate the options for conversation with NPCs and to display them in the same window. This is relatively straightforward, and I had to make only a few minor changes to Sand-Dancer itself to get the thing working.
Interfaces like this one have a clear benefit where players new to IF are concerned. Possibly they will be easier to navigate than the standard interface, but whether or not that is true, the presentation of full commands in buttons provides a mechanism for teaching the basic command grammar. Over time, clicking on buttons like “examine the letter”, “press the red button”, “ask the lawyer about copyright law” and so on will teach not only those individual commands, but the overall gestalt of the parser interface. Presumably this would lower the bar to playing unadorned parser IF.
As to the more focused question of whether the interface demo’ed here is a productive direction, I leave that to you. (I’m still too close to the micro-choices I’ve been making to evaluate the experience as a whole.) Please give it a shot and let me know in the comments how you found the experience. Even better, if you hold sway over someone with little IF experience, see what they think. (My wife says it solves most of the problems she has with IF–barring the map–but that’s just one data point!)
Final note: There are plenty of technical weaknesses in the implementation of the overlay interface, including both missing features and some bugs as well. For this reason, I am not releasing the source code as I normally would–I don’t want anyone to use this particular code as the basis for their own project.
Glimmr 
Comments on: "Sand-Dancer Sans Keyboard: An Experiment with Hyperlink Input in IF" (16)
I have nothing more useful to say than “wow, that’s really nifty.”
Unfortunately I only have a laptop, so I bet my perception of this is colored by hardware.
I think it looks really good, but the big sticking point for me was switching attention between the three text areas. I wonder how much that has to do with already being trained on ‘single-window’ IF.
I’m guessing more visual feedback would help with the reading flow — like highlighting when clicking inventory and switching attention to the upper right object area, and putting more emphasis on new vs. old text in the main window.
Thanks for taking a look!
@George
I don’t know whether your difficulties shifting attention between the different windows is do to habit/training or not, but I think you’re right that moving between the main and object-focused windows is the most ergonomically taxing part of the whole interface. (Apologies for hijacking “ergonomics” there–do interface designers have a better word for the analogous concept applied to attention?)
I would have really liked to be able to do something about old vs. new text in the main window. Personally, I think this has been somewhat broken in all of the IF interpreters up to this point: the text window just updates all at once, when smooth scrolling would really help to instruct your eye as to where the new content is. But it’s even worse in this example where you have multiple windows to keep track of. Overall, subtle animations to help direct the eye to the interrelations among the content areas would probably help with all of the things you suggested.
I think it’s a pretty neat implementation. I had never played Sand-Dancer before today. One thing I noticed was that with this user-friendly scheme, I did not even have to guess as to what I should do with the tin can. It was made blatantly obvious via the interface.
Interesting. I tried something a little similar to this a while back, but with a few more layers of complexity and it was just overwhelming. This is a lot nicer.
The only thing I don’t think I like is that clicking a noun – in the text, or the inventory window – causes *two* things to happen: an examine action, and the focus / context verbs in the top -right window. I feel like selecting a noun should simply shift the game’s attention to that object and generate it’s list of commands “for free”, without taking a turn … and then one of those commands is “examine.”
The most obvious time this will be important is in any kind of timed sequence but I think it might also help with the “paying attention to multiple windows” problem the other poster mentioned up-thread: in this scheme, a click on any link changes the window-top right; while a click on any command button changes the main game window.
The other thing this kind of demo really brings home is how many choices a parser-IF player – and author! – has to deal with every single turn. It is, by most game standards, rather a lot.
@Jizaboz
I’m glad you liked the way that action transparency works for the tin can. The tin can is actually an alternate solution–the jade is the primary object to be thrown (hinted in the description of the jade). For me it makes a nice progression that goes a long way toward ensuring that the player can get into the building without tearing his hair out:
Anything with a certain heft can be used to break the window. In the original Sand-Dancer, you can actually throw your jacket at the window to break it (that was an unintended oversight in the original design, I think).
@Jon
Early on, I actually had the examine command working more or less as you’ve suggested (it’s only one or two lines of code). Clicking on a link merely shifted the focus in the upper-right window to the object, without printing anything to the main window. I didn’t like this because it required multiple clicks for what was essentially the most commonly issued IF action, examining things. However, I think you’re right that it would the flow that much clearer… Maybe I’ll post an alternate version tonight, restoring that behavior, for comparison’s sake.
(The question of timed sequences is one that I happily toss to one side–in my book, actions like examine, look, and inventory should never count as taking time.)
On the question of the number of player/authors–yeah, it’s pretty wild. Especially since I left out many things that are doable but not really sensible. “Put on” and “put in” for example, don’t have much use in this game, so I left them out…
I have put up a second version of the demo, this time following Jon’s suggestion to streamline flow. Version 2 is at http://goo.gl/wsRxe.
Congratulations — this is pretty awesome! It’s a touch sluggish on a Samsung Galaxy, but I’m guessing further optimizations to I7 and Quixe ought to help with that. As a side achievement, your demo of this is now the best available IF solution for Android.
Wow from me too, very, very nice! Really looking forward to the complete release!
Unfortunately, as you predicted, it’s quite slow on my LG Optimus Black (note: that’s the phone without the dual-core processor!). I also got a lot of ‘red-banner’ errors when trying to enter text commands rather than click the pre-defined command buttons. Probably down to big fingers. But it seemed to struggle to know when to open the on-screen keyboard. However, it’s perfectly usable on my windows netbook and a lot more readable than on the phone, purely down to screen size.
I guess some tailoring could be done to consider tiny screens – where the pixels are tiny – and re-jig the layout to try to avoid scrolling and zooming but still display most of the basic info..
I’ll have to wait until Christmas to try out any sort of tablet, I’d better be good?!
This reminds me a lot of a similar demo that was done with Quest. The goal there is also an easy to use mobile device interface.
The use of hyperlinks rather than typing is the key to drawing the player in. Then players don’t have to fight with tiny on-screen keyboards and with the parser to be understood.
Is this going to support glimmr too? I guess so?
This look very well-implemented and it certainly makes a newcomer’s foray into IF much easier and pleasurable. But my mine gripe with getting rid of the keyboard is that by presenting possible actions in the right window, you are actually destroying the illusion of the ‘interactive’ in Interactive Fiction. Sure, it’s an illusion, but getting the player to mentally link possible action to the text and telling the game what your next action in going to be, is the whole reason play IF for anyway, isn’t it? The hyperlinks just don’t allow the player to immersive themselves in the story – the player will merely gloss over the non-hyperlinked text to get to what the game is telling them is important. Although definitely easier to use on mobile devices and great for newcomers (I can see this being used as a ‘help’ option incorporated into every game), I think it takes away most of the need to ‘think’ while playing. Congratulations on making a vey polished piece of work though! 🙂
I posted comments on Jon’s blog, http://threeedgedsword.wordpress.com/2011/11/28/choice-from-analogue-to-digital-and-back-again/ but I should have posted them here.
Contra Jon, I think the “click to examine” version 1 is much better than the “click to focus, second click to examine” version 2, but Jon’s right that clicking on objects tends to fill the transcript with spam descriptions.
Overall, I think the right fix is to have the transcript edit itself as you go. For a good example of this, check out Undum’s system at http://undum.com/games/tutorial.en.html
Clicking on a hyperlinked option hides the option menu and slides the content up. Whenever new text appears on the screen, it fades in from opacity 0 to 1. This is all done with a few lines of jQuery magic, which you can see here in “doTransitionTo”:
https://github.com/idmillington/undum/blob/master/games/media/js/undum.js#L1114
By editing the transcript as you play, it’s possible to eliminate the object panel and do it all in the transcript. Clicking on an object would examine it and display the options inline, but the options would fade out and slide up when you clicked on one, or if you clicked on something else. If this was your second time examining the object, the object description would fade out, too.
Similarly, I think the inventory panel could be removed by hyperlinking “you” in the transcript. Clicking “you” would examine yourself and display your inventory, and perhaps give access to other verbs that don’t take an object, like “wait.”
Like georgek, I had trouble with the multiple windows. I spent some time clicking on an object a few times before I realized where the verbs were, and worse yet, clicking on an action multiple times before I remembered to look back at the transcript.
I think I’d prefer it if clicking on an object brought up a pop-up menu with the verbs on it. It’d be nice if examining an object were privileged, but not in a way that auto-examines it whenever you want to do something with it; maybe double-clicking to examine? Forcing auto-examine and then making it disappear on the second and subsequent times seems like a bad compromise to me; sometimes the text changes the second time, sometimes you want to keep the text around because you last examined it a hundred turns ago, sometimes you just don’t want to see the text again, even if it fades right away. (Imagine you’re trying to put the object in five different places.)
Having the inventory appear when you hover over the bottom might be a good idea — it keeps it close to the text window. I don’t know if hyperlinking it on “you” would work. Inventory is important enough that you should always be able to find it in the same place, and that would also create a lot of nested menus (wait, I’m already assuming you adopted my pop-up menu suggestion). Plus it would break Slouching Toward Bedlam.
I like Jon’s suggestion of removing the hyperlinks and having the cursor change when you mouse over an implemented word.
All this nitpicking aside, excellent work!
Thanks to all for the comments!
@mwigdahl
Good news that it’s only “a touch” slow on the Galaxy. I take it that Android’s javascript engine must be a bit speedier than iOS’s. I feel like it’s just too slow to bother with on the iPad 2.
@veryalien
Yeah, I actually didn’t intend for anyone to try this on a mobile device, and particularly not on a phone-sized one. I see it more as a desktop mockup of the kind of interface you might look for on a 5-10 inch tablet. For a phone-sized interface, you’d definitely have to make some drastic changes.
I need to look at the Quest and Adrift web interfaces; I’m pretty ignorant of what they’ve come up with.
I hope this isn’t too disappointing, but I don’t plan to release any tools related to this demo. The playthings that I post on this blog under the “Experiment” heading are more like concretized musings than imminent releases. In the case of this sort of interface, mobile hardware, javascript, the Inform library, Glulx, and Quixe would probably all need to see significant speed advances before a straightahead gblorb + CSS example like this one would be workable. I think native apps are the only viable way forward for this kind of thing–probably FyreVM is the best place to start for games that come out of Inform 7.
@joe pereira
Thank you for the feedback! I agree that this kind of thing would work nicely for tutorials and hints. I also worry about whether the change to the interface impoverishes interaction, with players able to skip reading in favor of simply clicking on buttons. I’m not quite as pessimistic as you are about it, though: Players spend quite a bit of time skimming and ignoring text in standard IF interfaces as well, and it doesn’t ruin the experience. Games like Binary and The Play are played with hyperlinks, but I never found myself just clicking rather than reading. But only true playtesting with a fully developed example will really give us a handle on how this particular interface (or its offspring) might be experienced in the wild.
I guess I’m not really on board with the idea that the illusion of wide-open interactivity is key to IF. At one time I probably would have agreed with the sentiment at one time but … I just don’t think it’s true any longer. For me, Jon Ingold hits closer to the mark when he talks about the function of the command prompt in allowing us to ignore unneeded options, whereas an interface where commands are made explicit (such as this one) needs to find some other way to deal with the problem of overstimulation….
@Dan Fabulich
Thank you for taking the time to comment twice! On the question of “click to examine”, please check out version 3. It represents my original take on the problem, which I abandoned out of a (possibly misplaced?) concern for the integrity of the transcript. It allows the player to get object descriptions for free, but also doesn’t spam the transcript in the process. Like version 2, this version has an examine button, so you can get a description printed to the transcript by pressing that.
(I consider the main problem with the examine commands spamming the transcript to be, not the printing of the commands themselves, but the visual confusion of seeing two windows updated at once; I get the sense that this was Jon and George’s main complaint as well.)
Personally, I’m not too keen on getting rid of the inventory window, at least not in the case of desktops or tablets, where there is plenty of screen real estate available. Why make the player hunt for a hyperlinked “you” in order to interact with the things he carries?
Taking the Undum approach to commands in the main flow of text might work, though I would be worried about phone-sized screens. I’d like to see someone mock it up. (Unfortunately, I don’t think it would be trivial to add transcript editing to Quixe, as I don’t think Quixe tracks new vs. old output in any accessible way.)
@matt w
A pop-up menu of some sort would be the go-to option for a phone-sized version of the interface, I think–or at least it’d be the first thing I’d try. There’s no way to implement such a thing with a Glulx + CSS mockup like this one, unfortunately, since Glulx doesn’t support pop-ups. For full-size screens, though…I have to admit a personal dislike for pop-up menus. Somehow, they feel like work to me! I also feel like there would be more options for prioritizing commands if they were presented in their own permanent real estate; in a permanent window pane, you could swipe between first-order commands and second-order commands, for example, while it would seem odd to do that with a pop-up menu.
It’s probably important to point out that using hover to display content is not an option with touchscreen interfaces. Since my main goal here is to explore options for the mobile space, I feel like I need to simply disregard those suggestions. (Although I did incorporate hover-to-reveal-links into version 3 just to show how it would work in practice…)
But what is the problem with just having the inventory available all of the time? That seems to be a nice time-saver for the player, as well as reflecting the fact that the PC would likely have a good sense of what he was carrying at any given moment. I very much appreciated having the dedicated inventory window in Dead Cities, mostly because I find myself typing “i” about once every two minutes in a game that doesn’t have one. (This might be a personal quirk.)
Both the pop-up menu and the hovering inventory were meant to keep things closer together. I found it a little disconcerting that clicking on a link in the transcript made some verbs appear on the other side of the screen; I didn’t always even notice the changes. And the inventory thing was just meant as a way to keep the inventory options close to the command prompt, as a way of avoiding the “look over here then back here” problem. Having the inventory window at the bottom would serve just as well, I just thought it might be cluttered.
I hadn’t thought about the lack of hover on touchscreens! Maybe one approach would be to offer the player the option of either subtle highlighting or blatant highlighting. (I looked at zarf’s take on writing point-and-clicks for touchscreens, but the ideas aren’t really applicable.)
Hm, OK. I guess that the desire to get the input choices closer to the text input box is the common thread linking yours, Dan’s, and Jon’s suggestions. I’ll see what I can come up with, but I’m not convinced that I can do anything satisfactory without some major hacking on Quixe–and major hacking on Quixe I’m not really willing (and probably not competent) to do.
More generally: I don’t feel like looking to a different part of the window for information is a problem in itself. In other words, it might well be a problem that the verbs show up off to the right and away from the traditional input area, as you guys are suggesting. But, based on my own response to games like Dead Cities and Binary, I don’t think that having the inventory in another pane causes any difficulty.
[…] Light is also a descendant of the hyperlink-only web interface that was elaborated in prototypes here, here, and here. I beta-tested the game and I think that, in addition to being a good game, it […]