Journal (September 1996)

Previous Up Next

Okay, I'm making some good forward progress on the goal stuff. I've run into one problem that causes my stuff to completely blow up. Basically, it seems to have to do with building on existing chunks while I'm learning in the execution concept. I get into these god awful backtraces that just send my memory through the roof.

Okay, I've finished a first cut at integrating SFA and GIPS. I now use SFA to do all the precondition recall and execution concept training. Still have some bugs -- the one I mentioned above is the one that I'm most worried about.

I've got the thing running in count-abstract mode (like SCA2). Reason is, when you run it in feature abstract mode, it kinda blows up because I don't have a robust set of general rules that ("in a real setting") would have been built. At some point I need to cut it over to this so I can explore things a little more carefully, but not now. Also, I haven't checked how well the precondition learning works at all.

Looks like I'm still having some trouble communicating what to do between transforms and subtransforms, too.

Think I might have figured out what's causing my use of the "token pool" to go ballistic. Big big chunks, like 100-conditions big. I sent a note out to Soar-UM to see if anyone has any ideas, so we'll see. Of course, this problem will go away a little bit if I use a less aggressive learning algorithm.

Started hacking changes to Gips to get the SFA stuff in. First I went ahead and updated everything to version 2 so that if this blows up I can always fall back to the "straight SCA" junk. Reorganized the SFA code and replaced all of the "scc" tokens with "sfa" just to be neat. Tested the example code and it still works. Checked all that crap into RCS, too.

Allright. I've run into my first sort of conceptually difficult problem. Goals. Eventually, I need to have some kind of seamless interface between execution concepts and generation of subgoals. I think that I'll still have some kind of intermediate mechanism that processes the execution concept and creates a new "subgoal" chunk. At a minimum, my problem solver will try to do an iterative recall given the cue "action = execute", and will then explicitly cast each of these features into an "achieve" or "remove" subgoal.

Hmm...maybe easier said than done: how do you detect a consistent absence of a feature so that you know that you want to remove it? Maybe all features need to be constructed in the positive? Then when applying it to a new domain, how do you know which of these features should be constructed a priori? Well, I'm not going to worry about that, I'll just assume that I can get them. Remeber, we're just trying to demonstrate a problem solver that gradually converges on the correct concept.

Okay, I've almost got everything working -- just wanted to make a quick note so I can remember what I need to do next in case it's a while before I get back to this. Gotta get subgoal recall to work. This isn't going to be as easy as I first thought. I need it to do an iterative recall based on a cue: <op> ^status execute. Not rocket science, but still not a triviality.

Allright. I think I'm at a point now where I need to just do it and integrate my feature association stuff into the GIPS process. I've juggled a bunch of directories around, and am now trying to figure out what it's going to take to "make the cut."

1. Rewrite the MOVE-BLOCK selection concept chunks.
2. Add knowledge to GIPS training engine so that the operator is always the last feature removed; i.e., the target concept.
3. Rewrite the MOVE-BLOCK execution concept chunks.
4. Rewrite the "subgoal recollection" chunks to be done as FA chunks. Ideally, these should "naturally merge" with the execution concept chunks.
5. Rewrite the GIPS process that deals with recollection of chunks
6. Build a chunk charting tool. Ideally it would be something that could display the chunks as a tree.
7. Add a flag so that I can turn off learning. This'd be nice for testing.

So this morning I'm trying to write a set of Tcl utilities that will convert the SCA rules into a nice tree for me to look at. So far I've written the stuff that will rip out the features and the prediction from an SCA prediction chunk and print them out nicely. So now the part I have to figure out is, how do I actually fit these things into a tree?

I think that the features will have to correspond to edges in the tree, and the predicted value will have to correspond to vertices. I know that it's going to end up as a tree, not a graph. So I'm thinking that the way to construct this thing is to iterate up from the rules with the fewest conidtions to the rules with the most. One thing that's kind of funky is that I can't do exact matching (unless I can figure out some "canonical" way to represent the rules), because I need to bind variables to one another. For example:

 ^color red
 ^on-top-of 
-->
predict blech

Could be one rule, and that would need to "bind" with

 ^color blue
 ^color red
 ^on-top-of 
-->
predict blach

Actually, now that I think about it, I'm not even so sure that you'd end up with a tree. I think I'm pretty sure that you wouldn't if you used the colonel's original recipe for abstraction. Maybe with Doug's method you're okay, though.

Maybe we can assume that since the rules were all built incrementally, the bindings at the lower levels will all be the same, so simply sorting by the number of conditions in the rule is sufficient to give us the results we want? Nah. Forgot that Soar will reorder the conditions to optimize the RETE network.

So IF I remember correctly, I was having a hard time figuring out how to make sure that I could always solve problems. Specifically, I was ending up in some trouble with deep, deep subgoal stacks that I could never get myself out of.

• I have a rule that rejects predictions for operators that have already been tried in a superstate. This seems to end up giving me an infinite regress of subgoals, but now that I think about it, should we just end up terminating the transform unsuccessfully? Maybe this is a corner case that I haven't tried yet...

• I tried implementing a rule that would reject any execution concept predictions to SUBGOAL once the goal stack exceeds a certain depth. I thought that this was great because (1) it's pretty implausible that you'd be able to maintain a 36-level goal hierarchy in your head, and (2) it gives me a way to EXECUTE operators that I have no business executing in the first place. (There is this whole problem that I vaguely remember of "how do I simulate that little teeny weeny chance that you might execute.")

At this point, both of these approaches ended up with strange bugs, but it was late last night and my brain was fried. I'm now suspicious that what might be wrong is exactly the case where I exhaust all of the possible operators in a TRANSFORM. In this case, I need to make sure that the TRANSFORM terminates and signifies that it failed.

If I think about the first problem some more (rejecting operators that you've tried in a superstate), I think that I see why what I do isn't working right. The problem is I allow it to guess if it "bottoms out." Because of this, a TRANSFORM will never terminate unsuccessfully: it can always keep guessing. Well, at least until all the operators have been tried in some superstate. But then, when we return to that superstate, we've forgotten about all the operators that already failed. So we do the exact same thing all over again. And again and again and again. Sooooo.... what to do? I think that I need to be able to guess, otherwise my system will end up so scared that it will never be able to predict anything. Or, maybe I should back off on the learning algorithm and try to choose something a little less aggressive.

So I went ahead and removed the whole "guessing" thing, or at least put a flag on stuff to turn it on and off, and things seem to be working a *lot* better. There's still one bug that's nagging at me: seems like the TERMINATE-TRANSFORM(failure) in a SUB-TRANSFORM ends up crashing the system -- it just keeps selecting TERMINATE-TRANSFORM over and over again. Doesn't seem to make sense, because it looks like the operator termination conditions on SUB-TRANSFORM are right. I put it a hook there to halt it, but haven't been able to reproduce the problem yet.

So I'm in the middle of this nightmare test run that exhausted the VM on knine. I'm trying to kill it so that I can dump out the RETE net or at least get some statistics on memory usage, but to no avail. It's locked up tighter than a drum. Sucks because it was only about my fifth or sixth trial: if that's all that it takes to wipe out Soar, I'm in deep shit.

As an aside, how will this thing ever learn about interacting subgoals? For example, will it ever learn that if you want to put 1 on 2 and 2 on 3, that you need to put 2 on 3 before you put 1 on 2? I guess it would have to learn to select one before the other?

At some points, it did seem like it was learning this, but one can never be sure...

Re-arranged all the blocks world simulation code into a single flat directory and put it all into RCS so that I could keep my different sites in sync.

Started running it and all of a sudden I'm starting to get this tie impasse that I thought that I'd fixed a long time ago. Okay, I'm getting this operator tie during training for the selection concept -- I've got a REJECT prediction, and the system can't decide whether to select a feature and learn the new rule or to continue rejecting stuff.

Spent a couple of minutes trying to formalize the stuff that I've known about all along, just to make sure that I know what I'm doing. I guess that if I'm feeling mathy some day I've come up with some little lemmas to prove about how SCA works, what conditions have to hold for it to work properly, etc. Went back and took a look into the SCA code and it seems to me that a lot of stuff needs to be moved out of SCA and into operator-specific implementations. Basically, anything that relies on determining if a prediction has been made before. Since I allow for stuff on the LHS to get carried through to the RHS, then I need class-specific rules to determine whether a prediction was actually made. I remember spending (wasting?) a lot of time trying to see if I could automate this process by building some kind of recognition chunk, but I don't think that its going to be worth it.

Things keep working better and better (knock on wood). I found a couple more minor bugs. Excised the rule that keeps you from selecting an operator that you'd already tried in a superstate. Reason being, you can transform the state such that the operator can apply in a substate (even though you'd tried it before).

I think I stumbled across something pretty cool. Turns out the you can "overlearn" the execution concept for an operator so that you end up in an infinite regress of subgoals. So I added a hook in the code to check the depth of the goal statck, rejecting SUBGOAL if the stack gets too deep. Now what I hope is that this will also solve the problem of getting an operator to execute when you don't have any other reasons to execute it. In other words, in GIPS, there was always this chance that you might execute an operator even though you'd learned pretty thoroughly that you're supposed to subgoal. So what I'm hoping here is that this will give me both a way to avoid ending up with really deep subgoal stacks and a way to execute hopelessly "over subgoaled" operators.

Implementation is a bitch. I'm getting too tired to make sense of what I'm trying to do here, so I'm going to leave it where I've got some productions computing the depth, but I'm gonna toss out the production that rejects the subgoals.

Worked some more on getting Gipsosoar up and running. I think that I've breathed some life back into the thing. It's amazing how little that I'd actually tested this thing on: I think I got stuck in a rut with one test case and really didn't check to make sure that it worked across a wide variety of cases. Anyway, I'm now faced with the problem that I build an overly general REJECT preference that has the effect of rejecting all of the possible operators for selection. If I remember correctly, this was why I built the DEFAULT category of predictions so that I could "guess" if I ran out of ACCEPTABLEs.

So I think my next task is going to have to be to uncover how the DEFAULT preference is supposed to work and get it up and running again.

So I'm wading through my old "symbolic conceptual clustering" stuff (which I think now would be more aptly named "symbolic feature association"), and am trying to figure out where I was. It all working in Soar 7.0.4 with a few quick hacks to the code, but my comments were sparse and if I remember correctly, I was in the midst of a trauma about whether to use an aggressive or unaggressive learning mechanism. Looks like what I ended up deciding (here) was that implementing both within the same system was possible, and that doing the less aggressive "original SCA" mechanism seemed more promising.

So. I went back through the symbolic feature association code and it all seems to run pretty well. I'd put together a "weather" demo for it, and yup, it can learn to recall an instance from the training data set. Now what I haven't tested the thing on is noise and disjunctive concepts. What I was going to do (if I remember correctly) was to just have it recall the necessary conditions for those concepts; i.e., the conditions that we're not present in every non-instance of the concept. This should be pretty easy to deal with during recall, because all you need to do is to throw away anything that would give competing values for an attribute. Didn't I try to do this? I'll have to go look...

Next, tried to revitalize the old Gipsosoar/blocksworld stuff. I must have hacked the old working memory and feature vector stuff that I'd originally written a couple of months ago too much, because now the blocksworld stuff just dies mysteriously when it first starts. I fixed the few problems that were because of the move from 7.0.1 to 7.0.4, but what I've got now is outside of that. What was I working on so that I needed to hack up the working memory junk for anyway? Oh yeah, the SUM-to-MIN model...

So. After some mild modifications, I think that I have things a little bit more under control. Had to make sure that I set indifferent-selection to "random" so that it would randomly pick among alternatives. Now I seem to be getting a strange operator tie condition that I don't remember before, but I should hopefully be able to track that down.

Seems like I have some old bugs in the SCA code that I wrote -- or at least, I can't figure out what I meant. The biggest problem right now seems to stem from REJECT predictions. They create prediction structures like regular old ACCEPTABLE predictions do, but then there's no way for anyone to fire because I've got a rule wired up that says "don't predict anything if a prediction is already made!"

So yes, it looks like I've got a bug here. Shoulda tested it more, huh?

Think I buttoned up the security problems on the Soar web server -- of course I won't really know until something goes wrong. Ended up just removing the dangerous commands by "renaming" them to the null string. The people that I talked with didn't seem to think that there was any way to get them back once you've done this. Also polished up the java applet a little bit, making the window display a fixed-width font, moving some of the connect and disconnect code to the start() and stop() methods, etc.

Yesterday afternoon I began working on the Win32 port for Soar. I downloaded soar-7.0.3 (which was stupid, because I got an announcement yesterday for 7.0.4) and started hacking changes. Only this time, I kept a laundry list of stuff that I needed to do to get it to compile and link. So now I'm at a point where I can start Soar in Win32, but it dies pretty quickly because none of the callbacks are available. I sent a note to Karen saying, "whatdya want me to do", so we'll see what to do next. Also, Tom Head mentioned something about cleaning up the GUI code, so I don't know if someone else is already doing that stuff.

My research stuff has definitely fallen by the wayside in the past two weeks. I talked with Randy last Tuesday and we agreed that there were two directions that need to be pursued. First, is deciding how I want to construct the model. Do I want to use means-ends analysis and alter the preconditions of operators, or do I want to use hierarchical operator implementation, where I would alter the proposal conditions of an operator. The single biggest didffernce between Randy's implementation and mine is that in mine, the SUM to Shortcut-SUM strategy shift is cause be the alteration of the proposal condition on REPRESENT-ADDEND. In Randy's the shift is caused by alteration of the preconditions for his START-COUNT. In his problem solving, START-COUNT is part of the "implementation" for REPRESENT-ADDEND, by which I mean it is executed in a subgoal for REPRESENT-ADDEND.

Second, I need to figure out how to incorporate what I've done with the "symbolic conceptual clustering" stuff into the GIPS algorithm as a way to learn bi-directional rules. This is sort of a "bottom up" approach ("I've got a hammer so let's see what things look like nails!"), but the original problem that I was trying to solve oh so many moons ago was explicit representation of operator preconditions.

Been a while! Last week I finished setting up a little Soar web page that allows you to run the eight-puzzle demo using a Java applet. Not really anything great, the tough part was getting the server-side "C" code up and running and dealing with flushing all of the sockets, etc. Anyway, it'd probably be worth pursuing a more exciting user interface that actually interprets Soar's output and displays the results using a more advanced user interface. I'd thought about maybe doing the blocks world simulator in Java, but I think that'd be a bit more work than I'm willing to bite off right now.

One real bummer about this whole thing has been security. I have the server's setuid bit set, and its owned by "nobody", however, it still seems to be able to send mail as me. I think this must have something to do with the way it reads the "effective user ID" or something. Makes sense, because I can kill the processes that get spawned. Anyway, Frank recommended that I just unset all the commands, which I've done with the rename command in the soar.soar file. Seems to work from the command line, and seeing as the web server seems to be dead right now, I guess it's the best I can do.

Anyway, I need to start making some forward progress on fixing up the GUI-soar interface and the Win95 port.