GeeWhiz Prolog - Part Five - Adding The Compiler

Wednesday, May 21. 2008

NetBeans
This is Part Five, the sixth part of a series of entries describing an implementation of the Prolog language in the NetBeans IDE.
Part Zero, "About The Project," is here.
Part One, "Before We Start," is here.
Part Two, "Creating A File Type," is here.
Part Three, "Adding Language Support," is here.
Part Four, "A Visual Prolog Modeler" is here.


What We'll Be Doing


Up to this point we've created a Prolog editor within the NetBeans IDE with built-in syntax highlighting, navigation, a visual modeler, and some other stuff that I forget but that's probably pretty cool. Now we're going to add in-place compilation so we don't need to switch between NetBeans and whatever Prolog compiler we're using just to fix syntax errors and such.


In this entry I'll be using SWI Prolog as my Prolog compiler. It's a free, robust open-source compiler that runs on multiple platforms including, probably, yours. One thing it does not have, however, is a feature-rich IDE -- hence this project. When we're done with this segment, we'll be able to compile our Prolog source code directly from the NetBeans IDE and hotlink (whatever that means) the error lines back to our source code. In other words, it will function much like the Java compilation process within NetBeans that we all know and love.


By the way, if you currently use a different Prolog compiler, don't worry. Once we're done here it should be pretty obvious how to adapt this project to incorporate your favorite compiler as long as you can run it from a terminal command line. In fact part of the reason I'm writing this how-to is to allow you to adapt ANY language to the NetBeans IDE. I'm trying to assemble the knowledge that's scattered around the NetBeans universe in one place so that this series of articles gives you the ability to quickly and dirtily add your favorite language to the NetBeans IDE. We'll work on the "dirtily" part later; for now our goal is results, baby.


Adding Compiler Support


Adding compiler support to GeeWhiz turns out to be surprisingly easy. It's easy if you know where to look for info, that is. It took me for-effing-ever to find the information I needed to do this, but once I did, implementing support for an external compiler turned out to be almost trivial. But I digress. The first thing we need to do is create an action in our project to invoke the compiler. Right-click on our GeeWhiz project and select "New" "Action".



In the first screen of the New Action Wizard, once again select "Conditionally Enabled" and "User Selects One Node". Set the cookie class to "DataObject" since we want to only compile Prolog source files with our compiler. Hit the "Next" button.

On this wizard screen select "File" for a category (just for something different; we chose "Other" for our "ShowDiagram" action). (I've always wondered why it takes so long for this window to come up on my machine. I suspect that this is the place where NetBeans is sending encrypted information about my unspeakable habits to our shadowy overlords.) Uncheck "Global Menu Item" and check "Global Toolbar Button" (again, for something different). We're going to stick our toolbar button at the very end of the Build toolbar so select "Build" for the toolbar and "Profile Main Project... - HERE" as the position. Click "Next".



On this screen, enter "CompileProlog" for a class name and "Compile Prolog Program" for a display name. For icons, you'll need both a 16x16 and 24x24 icon in the same directory, called (in our case) compile16.png and compile24.png, otherwise the wizard will grumpily issue error messages until you do. You can use these if you'd like, another modified Black Star of Prolog. Right-click on them in your browser and save them to a directory that is not your project directory and let the wizard copy them. Fill in the path and file name of the 16x16 icon in the icon field in the wizard. Make sure your package node ("org.myorg.geewhiz") is filled in in the package field and click "Finish".


Note: in the last screenshot ignore the GeeWhiz2 reference. I didn't make screenshots of this process the first time around, probably because I didn't expect it to work the first time! I thought I'd have to do 8 or 9 trials to get it right like I did for the other stuff in this project. Gee whiz, I must be learning something....


Once the wizard creates our new action, you can close the icon files in the editor and add the following lines to the CompileProlog class, where it says "TODO: use DataObject": 

        PrologCompiler compiler = new PrologCompiler();
        compiler.compileDataObject(dataObject);




Save and close the file. Great. We have a toolbar action, now all we need to do is write a PrologCompiler class. So let's do that.


Create a new Java class called "PrologCompiler" and replace the empty class with:


public class PrologCompiler {
    DataObject dataObject;
    String fileName;
    
    public PrologCompiler() {
        this.dataObject = null;
    }
    
    public void compileDataObject(DataObject dObj) {
        ProcessBuilder procBuilder;
        Process process;
        Map<String, String> env;
//        File currDir;
        List<String> cmd;
        String line;
        InputOutput io;
        OutputWriter outputWriter;

        // TODO: should save file first if it's been modified
        
        // set DataObject and file name
        this.dataObject = dObj;
        File file = FileUtil.toFile(dObj.getPrimaryFile());
        fileName = file.getAbsolutePath();
        
        // get an output window tab
        io = IOProvider.getDefault().getIO("Prolog", false);
        io.select();
        outputWriter = io.getOut();

        // construct the SWI Prolog process command
        cmd = new ArrayList<String>();
        cmd.add("swipl");
        cmd.add("-c");
        cmd.add(fileName);
        
        procBuilder = new ProcessBuilder(cmd);
        procBuilder.redirectErrorStream(true);
        // also s/b able to merge it into OutputWriter
//        env = procBuilder.environment();
//        env.put("VAR1", "myValue");
//        env.remove("OTHERVAR");
//        env.put("VAR2", env.get("VAR1") + "suffix");
//        currDir = procBuilder.directory();
//        if (currDir != null) {
//            System.out.printf("Current directory is %s.", currDir.toString());
//        }
//        procBuilder.directory(new File("myDir"));
        try {
            process = procBuilder.start();
            InputStream is = process.getInputStream();
            InputStreamReader isr = new InputStreamReader(is);
            BufferedReader br = new BufferedReader(isr);
            // TODO: might want to clear the output window first...
            outputWriter.printf("Output of running %s is:\n\n", cmd.toString());
            while ((line = br.readLine()) != null) {
                outputWriter.println(line);
            }
            // TODO: close outputwriter
        } catch (IOException ex) {
            Exceptions.printStackTrace(ex);
        }
    }
}


What we want this to do is invoke the SWI Prolog compiler with our source file, the name of which we retrieved from our trusty Prolog DataObject. We're using the IOProvider/OutputWriter we saw last time to write the output to the Output tab, just like a real adult language module does. We're also using ProcessBuilder to build a Process so we can execute an external command.


Just a couple of other things worth mentioning about the above class: one is that I left in some comments showing how to access the environment and change the working directory, just in case you need the info to make your compiler work. Also, note that the command line I used is for the Linux version of SWI Prolog. If you run on another platform, you'll probably need to change this. For example, I believe Windows uses "plwin". See the SWI Prolog site for more information.


Testing The First Version


So let's try it: clean and build your main project and install to the target IDE. Open up the BraveNewWorld project. Wait until the excitement dies down. Open up sieve.pro. Add a line somewhere in the program that says:


	gruesome(ugly error).


This introduces an error in our otherwise nearly pristine Prolog program. Save sieve.pro. Find our new compiler button on the toolbar and click it.

At this point my original notes say "holy shit it worked!" As indeed it did, the first time. Nothing works the first time, ever. Imagine my elation. If you look at the "Prolog" output tab (!), you'll see the gruesome ugly error we introduced does indeed give us an error in the compiler output. Also note in passing that our syntax parser did flag our gruesome ugly error statement as an error. Heh heh.


The compiler also gives us a warning that there is a singleton variable in one of the clauses. This means that the variable "P" in the statement "remove(P,[],[])." isn't used. In Prolog, instead of using a named variable here you should use an anonymous variable, the underscore. If you want you can change the statement to read "remove(_,[],[])." and recompile it to see that the warning goes away. When you're done muttering "Gee whiz!" you can close the target IDE.


Add Hotlinks (Whatever They Are)


Now we're going to add hyperlinks to our compiler output so when you click an error line in the output you are magically transported to the source code at the point of the error. I know, you're saying "Gee whiz!" to yourself again. Just wait....


In trying to add this feature, I knew exactly what I wanted to do but I really wasn't sure how to accomplish it. Until, that is, I found an article called "Meet A NetBeans Module Writer: Jens Trapp". In the article is a code snippet that does exactly what I wanted to do in about three lines of code! Thank you Mr. Trapp; may the rest of your days be happy ones. Who knew there was a LineCookie that plunked you right down in the middle of the source code editor? Probably the people who RTFD, that's who.... But for the rest of us there's Hulles.


So now we're going to change our PrologCompiler class to "hotlink" our compiler code to our source code. Open the class file and replace the class with this:


public class PrologCompiler {
    DataObject dataObject;
    String fileName;
    
    public PrologCompiler() {
        this.dataObject = null;
    }
    
    public void compileDataObject(DataObject dObj) {
        ProcessBuilder procBuilder;
        Process process;
        Map<String, String> env;
        List<String> cmd;
        String line;
        InputOutput io;
        OutputWriter outputWriter;

        // TODO: should save file first if it's been modified
        
        // set DataObject and file name
        this.dataObject = dObj;
        File file = FileUtil.toFile(dObj.getPrimaryFile());
        fileName = file.getAbsolutePath();
        
        // get an output window tab
        io = IOProvider.getDefault().getIO("Prolog", false);
        io.select();
        outputWriter = io.getOut();

        // construct the SWI Prolog process command
        cmd = new ArrayList<String>();
        cmd.add("swipl");
        cmd.add("-c");
        cmd.add(fileName);
        
        procBuilder = new ProcessBuilder(cmd);
        procBuilder.redirectErrorStream(true);
        // also s/b able to merge it into OutputWriter
        try {
            process = procBuilder.start();
            InputStream is = process.getInputStream();
            InputStreamReader isr = new InputStreamReader(is);
            BufferedReader br = new BufferedReader(isr);
            // TODO: might want to clear the output window first...
            outputWriter.printf("Output of running %s is:\n\n", cmd.toString());
            while ((line = br.readLine()) != null) {
                if (lineIsNotable(line)) {
                    outputWriter.println(line, listener);
                } else {
                    outputWriter.println(line);
                }
            }
            // TODO: close outputwriter
        } catch (IOException ex) {
            Exceptions.printStackTrace(ex);
        }
    }

    private OutputListener listener = new OutputListener() {
        public void outputLineAction(OutputEvent ev) {
            String outputLine;
            Line editorLine;
            LineCookie lCookie;
            int lineNumber;
            int columnNumber;
            
            outputLine = ev.getLine();
            lineNumber = parseLineNumber(outputLine);
            columnNumber = parseColumnNumber(outputLine);
//          System.out.printf("Showing line %d column %d.\n", 
//                      lineNumber, columnNumber);
            lCookie = (LineCookie) dataObject.getCookie (LineCookie.class);
            editorLine = lCookie.getLineSet ().getOriginal (lineNumber-1);
            editorLine.show (Line.SHOW_GOTO, columnNumber);

            StatusDisplayer.getDefault().setStatusText("Fix me!");
        }
        public void outputLineSelected(OutputEvent ev) {
            // Let's not do anything special.
        }
        public void outputLineCleared(OutputEvent ev) {
            // Leave it blank, no state to remove.
        }

    };
    
    private boolean lineIsNotable(String line) {
        boolean result = false;
        
        if (line.startsWith("Warning:") || line.startsWith("ERROR:")) {
            result = true;
        }
        return result;
    }
    
/*
SAMPLE SWI PROLOG ERROR AND WARNING LINES:
ERROR: /home/.../bravenewworld/sieve.pro:16:0: Syntax error: Operator expected
Warning: (/home/.../bravenewworld/sieve.pro:18): Singleton variables: [P]
*/
    
    private int parseLineNumber(String line) {
        int fpos;
        int startpos;
        int lineNumber;
        Character ch;
        StringBuffer numBuf;
        
        fpos = line.indexOf(this.fileName);
		// assumes there are no colons in file name!
        startpos = line.indexOf(":", fpos);
        if (startpos == -1) {
            return 1;
	// in caller, lineNumber is converted to lineNumber - 1,
	//    this causes an exception if this is 0
        }
        numBuf = new StringBuffer();
        for (int i = startpos+1; i < line.length(); i++) {
            ch = line.charAt(i);
            if (Character.isDigit(ch)) {
                numBuf.append(ch);
            } else {
                break;
            }
        }
        lineNumber = Integer.parseInt(numBuf.toString());
        return lineNumber;
    }
    
    private int parseColumnNumber(String line) {
        int fpos;
        int startpos;
        int colNumber;
        Character ch;
        StringBuffer numBuf;
        int colstart = -1;
        
        fpos = line.indexOf(this.fileName);
		// assumes there are no colons in file name!
        startpos = line.indexOf(":", fpos); 
        if (startpos == -1) {
            return 0;
        }
        for (int i = startpos+1; i < line.length(); i++) {
            ch = line.charAt(i);
            if (Character.isDigit(ch)) {
                // skip line number
            } else {
                colstart = i;
                break;
            }
        }
        if ((colstart == -1) || (line.charAt(colstart) != ':')) {
            return 0;
        }
        numBuf = new StringBuffer();
        for (int i = colstart+1; i < line.length(); i++) {
            ch = line.charAt(i);
            if (Character.isDigit(ch)) {
                numBuf.append(ch);
            } else {
                break;
            }
        }
        colNumber = Integer.parseInt(numBuf.toString());
        return colNumber;
    }
}


As you can see, we added an OutputListener that does the work of popping us into the editor at exactly the right place where the error is located. The parsing of the error / warning lines to get the line number and column number is ugly, but that's because A) I didn't want to have to relearn regular expression syntax for the umpteenth time, and B) I didn't have 244 Swiss Francs for the ISO Prolog error message standard so I was winging it from the SWI Prolog output. Actually, there are two ISO Prolog standards documents, so it would have been 488CHF. As if.


I suppose I should also mention the StatusDisplayer line. This is one of those things that I couldn't resist putting in but that gets obnoxious very quickly. When you go to an error line, it displays "Fix me!" on the status line at the bottom of the IDE. Hah hah. So take it out, you don't like it.


Testing The Final Version


Okay, let's give 'er a spin. Clean and build your project then install it to the target IDE as before. Open up the BraveNewWorld project. Wait. Open up sieve.pro. Unfix the error and the warning if you fixed them last time so we have something to work with. Click our "Compile Prolog" button on the toolbar.

We have hyperlinks! Click on an unlinked (white) line in the Prolog output tab to make that window current. Click on one hyperlink, then the other. Notice how the cursor is moved to the editor window to the line and column where the error occurred. Now leave sieve.pro open and open up fib1.pro. Introduce an error into the code, save it, and compile it with our shiny new button. The error hyperlink from that compile should take you to the error in fib1.pro. Now scroll up the Prolog output window to our first compile and click one of the original hyperlinks. Notice that sieve.pro is reselected and the error position is displayed again. Go ahead, say it, you know you want to: "Gee whiz!"


So now we can compile our little Prolog hearts out in the NetBeans IDE. We still need to open SWI Prolog to run the stuff, but that's not so bad. It should be possible to connect up a Reader in the same way we did with the OutputWriter to allow the interaction, however. Hmmm.... You're welcome to do that if you want. I haven't yet, but I'm sure it's only a matter of time before I have to try it. If you do, let me know how it comes out.


We now have a pretty decent Prolog IDE embedded within the NetBeans architecture. What's next? We're going to create a Prolog project template so we don't have to stick our lovely, elegant and erudite nonprocedural Prolog source code into an icky stinky Java class library, that's what's next. Stay tuned.


Files From This Entry



Posted by Hulles in NetBeans at 20:46 | Comments (0) | Trackbacks (0)

GeeWhiz Prolog - Part Four - A Visual Prolog Modeler

Tuesday, May 20. 2008

NetBeans
This is Part Four, the fifth part of a series of entries describing an implementation of the Prolog language in the NetBeans IDE.
Part Zero is here.
Part One is here.
Part Two is here.
Part Three is here.


What We'll Be Doing


So far we've created an editor for Prolog source code within the NetBeans IDE that supports navigation and syntax highlighting. Most of the previous stuff we've done can be found in other tutorials, but now we're going to branch out from there and explore some unknown territory. Specifically, we're going to build a visual predicate mapper for Prolog that will look like this when it's done:



Frankly, I'm not sure how useful the mapper really is in actual Prolog programming, but so far I have used it to find missing references in Prolog code I wrote and to get an overview of code others have written. For these use cases the diagram thing really does come in handy.


My starting point for this segment of the project was the excellent tutorial, "A Visual Database Explorer for NetBeans". Thanks to Toni Epple for providing a lot of useful information. I should also confess here that I'm far from an expert in the NetBeans Visual Library, but learning more about it is high on my to-do list. The library so far seems easy to use, featureful and well worth investigating further.


Creating A First-Cut Diagram


The first thing we'll do is create a static sample diagram in GeeWhiz using techniques described in the Visual Database Explorer tutorial, so let's create a new action in the project. This action will add an item to the main menu that opens our diagram window. Right-click on GeeWhiz in the Projects tab and select "New" "Action".

This starts the New Action Wizard. In the first screen of the wizard,

for action type click "Conditionally Enabled" and select "DataObject" from the drop-down box. This tells the wizard that we want our diagram associated with our Prolog DataObject, since it isn't relevant to other file types like Java source. Make sure "User Selects One Node" is checked, then click "Next".

In the next wizard screen, select "Other" for a category then check "Global Menu Item". For menu, select "View", and for position select "<separator> - HERE - Web Browser". Also check the "Separator After" box, then click "Next".


In the next screen of the wizard,

enter "ShowDiagram" for the class name and "Show Prolog Diagram" for the display name. For an icon, we're going to use a variation of the Black Star of Prolog that looks like this. Depending on your browser you can probably right-click this image and save it on your computer somewhere other than in your project, then enter that path and file name in the Icon field of the wizard window. The wizard will copy the icon into the project. Make sure the package line is filled in with your package, then click the "Finish" button. Wait for NetBeans to add the new action to our project, then close any open documents.


Next we need to create a TopComponent, which is essentially a top-level window in the NetBeans IDE. Right-click the project again and select "New" "Window Component". This starts the New Window Wizard, as you might expect.

In the wizard, use VProlog for a class name prefix and this icon. It is a copy of the icon we used earlier, so if you want you can just copy it on your computer. Enter the icon path and file name in the wizard and it will copy it into your project. Again, make sure the package name is filled in then click "Finish". The IDE will generate a bunch of files for your project, all of which you can close for now.


Now we need to edit some files, but first we need to add the Visual Library to our project. Right-click on the GeeWhiz project, select "Properties", and select "Libraries" as the category. To the right of the Module Dependencies window click the "Add" button to bring up the Add Module Dependencies window.

Find "Visual Library API" and select it, then click "OK". Note that this differs from the Visual Database Explorer tutorial because the Visual Library API is included in NetBeans 6.1 where it wasn't in version 5.5 used in the tutorial. Now your project libraries should look like this.

Click "OK" to close the project properties window. Next, open the ShowDialog.java file and replace the performAction method with this code:


    protected void performAction(Node[] activatedNodes) {
        DataObject dataObject;
        VPrologTopComponent win;
        
        dataObject = activatedNodes[0].getLookup().lookup(DataObject.class);

        win = VPrologTopComponent.findInstance();
        win.open();
        win.requestActive();
 //       win.loadProlog(dataObject);
    }


Notice that the last line is commented out. We'll add this back in later. Right-click in the class and select "Fix Imports" to fix up the import statements. Save and close the file.


Now we'll edit the top component. Open VPrologTopComponent.java. In the design view, click on "Scroll Pane" in the Swing Containers palette and drop it onto the frame. Make it fill the frame. Then click "Panel" in the palette and drop it onto the JScrollPane you just created. Right-click the JPanel and change the layout to BorderLayout. Now you should have a design view that looks like this.

Switch to source view and scroll down to the getPersistenceType method and change PERSIST_ALWAYS to PERSIST_NEVER. Save and close the file.


Now we're going to create a class to render the diagram. Create a new Java class called "VPrologGraphScene" and replace the empty class with the following code:


public class VPrologGraphScene  extends VMDGraphScene {
    private static final Image IMAGE_NODE = 
            Utilities.loadImage ("org/hulles/geewhiz/node.png"); // NOI18N
    private static final Image IMAGE_EXTERNAL = 
            Utilities.loadImage ("org/hulles/geewhiz/external.png"); // NOI18N
    private static final Image IMAGE_ITEM = 
            Utilities.loadImage ("org/hulles/geewhiz/item.gif"); // NOI18N
    private static int pinID = 1;
    private static int edgeID = 1;

    /** Creates a new instance of VPrologGraphScene */
    public VPrologGraphScene() { // demo
        createNode (this, 100, 100, IMAGE_NODE, "Clause1", "Internal", null);
        createPin (this, "Clause1", "start", IMAGE_ITEM, "Start", "Element");
        createNode (this, 400, 100, IMAGE_NODE, "Clause2", "External", 
                    Arrays.asList (IMAGE_EXTERNAL));
        createPin (this, "Clause2", "ok", IMAGE_ITEM, "okCommand1", "Command");
        createEdge (this, "start", "Clause2");
        createEdge (this, "ok", "Clause1");
    }
    
    private void createNode (VMDGraphScene scene, int x, int y, Image image, 
            String name, String type, List<Image> glyphs) {
        VMDNodeWidget widget;
        
        widget = (VMDNodeWidget) scene.addNode (name);
        widget.setPreferredLocation (new Point (x, y));
        widget.setNodeProperties (image, name, type, glyphs);
        scene.addPin (name, name + VMDGraphScene.PIN_ID_DEFAULT_SUFFIX);
    }

    private void createPin (VMDGraphScene scene, String nodeID, String pinID, 
                        Image image, String name, String type) {
        VMDPinWidget pinWidget;
        
        pinWidget = (VMDPinWidget) scene.addPin (nodeID, pinID);
        pinWidget.setProperties (name, null);
    }

    private void createEdge (VMDGraphScene scene, String sourcePinID, 
                       String targetNodeID) {
        String localEdgeId;
        
        localEdgeId = "edge" + VPrologGraphScene.edgeID ++;
        scene.addEdge (localEdgeId);
        scene.setEdgeSource (localEdgeId, sourcePinID);
        scene.setEdgeTarget (localEdgeId, targetNodeID + 
                VMDGraphScene.PIN_ID_DEFAULT_SUFFIX);
    }
}

Fix imports (I love that thing!) then save and close the file. Next we need to add some icons to our project. The easiest way to do this is to download one of the project files at the end of this article and unzip / untar the archive to get the icons. Copy them directly into your project's /src/org/myorg/geewhiz directory.


As a final step, we're going to go into Bundle.properties in our project then right-click on one of the properties and select "Edit". This file is where descriptions and such are kept. Edit the descriptions of the properties as follows (or make up your own):


CTL_ShowDiagram=Show Prolog Diagram
CTL_VPrologAction=Prolog Diagram
CTL_VPrologTopComponent=Prolog Diagram Window
HINT_VPrologTopComponent=This is a diagram of the current Prolog program


Testing The First-Cut Diagram


Now it's time to test our changes. Clean and build the main project then install it into the target IDE (see earlier steps for instructions on doing this). When the installation completes and your personal domain is created, open the BraveNewWorld project. Let it index its little heart out then open fib1.pro.


As a first step in testing, make sure everything that worked before still works. Click in fib1.pro and make sure the Navigator syncs with the editor pane and that the syntax is colored as it was before. Now go up to the menu and click "View". Our new menu item, "Show Prolog Diagram" should be there and be enabled, complete with icon.

Click on the selection. If it's not enabled, don't panic, you should still be able to click it. The selection should be contextually enabled (that is, it should be grayed-out when you don't have a Prolog file open and lit up when you do) but in my experience the contextual lighting up / graying out is a little iffy. I've always still been able to click on it to open the window though. You should get a window that looks like this.

Spiffy, eh? Experiment with moving the nodes around on the diagram and clicking the show/hide arrow on the nodes. Behold the awesome power of Visual Library VMD. When you're done, close the window. Now go to "Window" in the main toolbar and select "Prolog Diagram".

This should bring up the same sample window we saw a moment ago. This is VPrologAction.java functioning, the default action created when we made our top component. Eventually we'll get rid of it, but for testing we'll let it be for a while. When you're done testing / playing, close the target IDE.


Adding Prolog Structure Analysis To The Diagram


Now that we can display a sample visual model, let's add Prolog support to the project. First we're going to create a Java data structure class to represent a Prolog clause and its attributes. Create a new Java class off your package node in the project and call it PrologClause. Replace the empty class with this:


public class PrologClause {
    private String name;
    private Integer arity;
    private List<PrologClause> body;
    private StringBuffer textBuffer;
    private Integer instanceCount;

    public PrologClause(String name, Integer arity) {
        this.name = name;
        this.arity = arity;
        this.body = new ArrayList<PrologClause>();
        this.textBuffer = new StringBuffer();
        this.instanceCount = 1;
    }

    public Integer getInstanceCount() {
        return instanceCount;
    }

    public void incrementInstanceCount() {
        instanceCount++;
    }

    public void setInstanceCount(Integer instanceCount) {
        this.instanceCount = instanceCount;
    }

    public String getText() {
        return textBuffer.toString();
    }

    public void setText(String text) {
        this.textBuffer = new StringBuffer(text);
    }
    
    public void appendText(String text) {
        this.textBuffer.append(text);
    }
    
    public void setBody (List<PrologClause> body) {
        this.body = body;
    }
    
    public void appendBody(PrologClause clause) {
        body.add(clause);
    }
    
    public Integer getArity() {
        return arity;
    }

    public List<PrologClause> getBody() {
        return body;
    }

    public String getName() {
        return name;
    }
    
    public static PrologClause findClause(List<PrologClause> clauses, 
                       PrologClause clause) {
        for (PrologClause c : clauses) {
            if (c.getName().equals(clause.getName())) {
                if (c.getArity().equals(clause.getArity())) {
                    return c;
                }
            }
        }
        return null;
    }
}


Fix imports and save it. The class we just created is pretty straightforward. A Prolog clause is identified by its name and its arity. Arity is a Prolog term that means argument count. All clauses with the same name and same arity are considered part of a single predicate, traditionally denoted as name/arity. Thus, all the clauses in our fib1.pro code are instances of a single predicate, fib/2. In our PrologClause class, we're storing the name, arity, text body, and a count of clauses that comprise the predicate in the source code. See what's wrong? I misnamed the class -- it should have been PrologPredicate, since we're not generating an instance for each clause, only for each predicate. I promise I'll go back and change it One Day Soon. But for now, let's go on. You will also see in this class a list of clauses mysteriously called body. This is another unfortunate name, but what it represents is a list of predicates that are "called" by this predicate in one or more of its clauses, either in the arguments to the clause or in the body of the clause itself.


To give you an example, take the following nonsense Prolog source code file: 

	some_list_function(A,B,C) :-
		another_function(A,B),
		do_something_else(B,C).
	some_list_function(param_function(A,D),B,C) :-
		B is C.

This would generate one PrologClause object, with a name "some_list_function", an arity of 3, an instance count of 2 (there are two clauses), the text that is typed above, and a body list consisting of another_function/2, do_something_else/2, and param_function/2. As you can see, the body list resembles a list of functions called by this predicate.


Now we need some code that creates the PrologClause objects we just defined. Create a new Java class and call it PrologAST. Copy the following code into it:


public class PrologAST {
    private ASTNode rootNode = null;
    private DataObject dataObject;
    private OutputWriter outputWriter;
    private static ASTNode saveNode;

    public PrologAST(DataObject dataObject) {
        InputOutput io;

        // get an output window tab using
        // starting example from I/O API javadoc
        io = IOProvider.getDefault().getIO("Prolog", false);
        io.select();
        outputWriter = io.getOut();

        this.dataObject = dataObject;
        refresh();
    }

    private void refresh () {
//  TODO: put this in a Runnable?
        ASTNode root;
        
        root = getASTRoot ();
        if (root == rootNode) {
            return;
        }
        rootNode = root;
    }

    private ASTNode getASTRoot() {
        EditorCookie eCookie;
        Document doc;
        ParserManagerImpl pImpl; 
       
        eCookie = dataObject.getLookup ().lookup(EditorCookie.class);
        if (eCookie == null) {
            return null;
        }
        doc = eCookie.getDocument ();
        if (doc == null || !(doc instanceof NbEditorDocument)) {
            return null;
        }
        pImpl = ParserManagerImpl.getImpl(doc);
        if (pImpl == null) {***REF(pl.gif)
            return null;
        }
        return pImpl.getAST ();
         
     }

    public List<PrologClause> getClauses() {
        List<PrologClause> clauses;
        PrologClause clause;
        PrologClause existingClause;
        String fName;
        Integer fArity;
        List<ASTItem> children;
        ASTNode node = null;
        String nodeName;
        ASTNode bodyNode;
        String statementText;
        
        clauses = new ArrayList<PrologClause>();
        children = rootNode.getChildren();
        if (children == null) {
            return clauses;
        }
        dumpItems(children);
        for (ASTItem item : children) {
            if (item instanceof ASTNode) {
                node = (ASTNode) item;
                nodeName = node.getNT();
                if (nodeName.equals("Statement")) {
                    statementText = node.getAsText();
                    bodyNode = node.getNode("ListOfStructures");
                    clause = getClause(node);
                    if (clause == null) {
                        continue;
                    }
                    existingClause = PrologClause.findClause(clauses, clause);
                    if (existingClause == null) {
                        clause.setText(statementText);
                        addReferences(clause, saveNode);
                        if (bodyNode != null) {
                            addReferences(clause, bodyNode);
                        }
                        clauses.add(clause);
                    } else {
                        existingClause.appendText(statementText);
                        addReferences(existingClause, saveNode);
                        if (bodyNode != null) {
                            addReferences(existingClause, bodyNode);
                        }
                        existingClause.incrementInstanceCount();
                    }
                }
            }
        }
        dumpClauses(clauses);
        return clauses;
    }
    
    private PrologClause getClause(ASTNode node) {
        PrologClause clause;
        ASTNode functionNode;
        ASTNode functorNode;
        ASTNode argNode;
        ASTToken token;
        String functor;
        Integer arity;

        functionNode = findChildWithName(node, "Function");
        if (functionNode == null) {
            return null;
        }
        functorNode = functionNode.getNode("Functor");
        if (functorNode == null) {
            return null;
        }
        token = functorNode.getTokenType("identifier");
        if (token == null) {
            // built-in function
            return null;
        }
        argNode = functionNode.getNode("ListOfStructures");
        saveNode = argNode;
        if (argNode == null) {
            return null;
        }
        dumpNode(functorNode);
        dumpItems(functorNode.getChildren());
        functor = functorNode.getAsText();
        arity = getArity(argNode);
        clause = new PrologClause(functor, arity);
        return clause;
    }
    
    private void addReferences(PrologClause clause, ASTNode node) {
        List<ASTItem> children;
        PrologClause embeddedClause;
        ASTNode child;
        
        children = node.getChildren();
        if (children == null) {
            return;
        }
        for (ASTItem item : children) {
            if (item instanceof ASTNode) {
                child = (ASTNode) item;
                embeddedClause = getClause(child);
                while (embeddedClause != null) {
                    if (PrologClause.findClause(clause.getBody(), 
                            embeddedClause) == null) {
                        // only need one per customer
                        clause.appendBody(embeddedClause);
                    }
                    embeddedClause = getClause(saveNode);
                }
            }
        }
        return;
    }

    private ASTNode findChildWithName(ASTNode node, String name) {
    // find the (depth-first search) FIRST ASTnode with name in node's
    //  descendants
        List<ASTItem> children;
        ASTNode child;
        ASTNode retNode = null;
        
        children = node.getChildren();
        if (children == null) {
            return null;
        }
        for (ASTItem item : children) {
            if (item instanceof ASTNode) {
                child = (ASTNode) item;
                if (child.getNT().equals(name)) {
                    return child;
                }
                retNode = findChildWithName(child, name);
                if (retNode != null) {
                    return retNode;
                }
            }
        }
        return null;
    }
    
    private void dumpItems(List<ASTItem> items) {
        ASTNode node;
        
        for (ASTItem item : items) {
            if (item instanceof ASTNode) {
                node = (ASTNode) item;
                dumpNode(node);
            } else {
/*                
                outputWriter.println("Item: " + item.toString());
                outputWriter.println("\tclass: " + item.getClass());
                outputWriter.println();
 */
            }
        }
    }
    
    private void dumpNode(ASTNode node) {

        outputWriter.println("Node: " + node.getNT());
        outputWriter.println("\ttext: " + node.getAsText());
//        outputWriter.println("\tprint: " + node.print());
        outputWriter.println();
    }
    
    private Integer getArity(ASTNode node) {
        Integer arity = 0;
        List<ASTItem> children;
        ASTNode child;
        
        children = node.getChildren();
        if (children == null) {
            return null;
        }
        for (ASTItem item : children) {
            if (item instanceof ASTNode) {
                child = (ASTNode) item;
                if (child.getNT().equals("Structure")) {
                   arity++;
                }
            }
        }
        return arity;
    }

    private void dumpClauses(List<PrologClause> clauses) {
        List<PrologClause> eClauses; // embedded clauses
        for (PrologClause c : clauses) {
            outputWriter.println("Predicate:" + c.getName());
            outputWriter.println("Arity:" + c.getArity());
            outputWriter.println("Instances:" + c.getInstanceCount());
            outputWriter.println("Text:" + c.getText());
            eClauses = c.getBody();
            for (PrologClause e : eClauses) {
                outputWriter.println("Embedded Predicate:" + e.getName() + 
                        "/" + e.getArity());
            }
            outputWriter.println();
        }
    }
}


This is a obtuse little class, but in general what it does is search the AST for Prolog clauses and deconstructs them into predicates, then builds PrologClauses for them if they don't exist yet or increments the count if they do. Text for each clause in a predicate is added to the PrologClause, and any foreign clauses called with the current clause are added to the body list. When we're done searching the AST we have a nice little group of predicates ready to diagram.


Note: this AST is of course the same one that appears when you click "AST View" in "Window/Other" that we looked at earlier. If you want to compare the text output of PrologAST to the AST view of fib1.pro, the code makes a lot more sense. Plus you can see how I was able to write the class in the first place.



A couple of things in PrologAST are worth examining more closely. The IOProvider/OutputWriter stuff has not been introduced yet in this series, but what it does is create a "Prolog" output tab and writes to it. Technically I shouldn't be hanging on to the OutputWriter by declaring it at the class level, but it works for now and I'll probably end up getting rid of the text output entirely anyway. When I'm sure it works correctly. Probably the same day I rename PrologClause to PrologPredicate.


Another thing to note is the ParserManagerImpl class in getASTRoot. I needed to obtain the root AST node somehow, and the only way I could readily find is by using ParserManagerImpl, which unfortunately is not part of the API. This one line causes us to use an implementation version of the GLF, a process I have to describe below. If anyone knows of an API way to get the root AST node, please let me know so I can get rid of the versioning and use the public API. There is essentially no documentation for the whole GLF, so finding something like that is a non-trivial task. Trust me.


To get rid of the red squiggles in the source code, we need to add some module dependencies to our project. Right-click on the project, select libraries, and click "Add Module Dependencies". Add the "Editor," "Editor Library," and "IO APIs" libraries. (You have to add them one at a time.) Now, as mentioned above, we need to change the Generic Library Framework to use an implementation version, as opposed to the public API. In your project, find "Generic Libary Framework" in "Libraries", right-click it and select "Edit".

Click "Implementation Version", type "0" in the Major Version, and click "OK". Now PrologAST should be compilable once you fix imports again. Whew.


Now we need to edit VPrologGraphScene to use our new predicate objects. Open it up and add the following code right below the first constructor:


    public VPrologGraphScene(DataObject dObj) {
        PrologAST pTree;
        List<PrologClause> clauses;
        List<PrologClause> embedded;
        String nodeID;
        Integer instances;
        String newPinID;
        PrologClause existingClause;
        String eNodeID;
        List<String> eNodes;
        
        pTree = new PrologAST(dObj);
        clauses = pTree.getClauses();
        // create all primary nodes first so they're available
        //   to create edges to....
        for (PrologClause clause : clauses) {
            instances = clause.getInstanceCount();
            nodeID = makeNodeID(clause);
            createNode(this, randXPoint(), randYPoint(), IMAGE_NODE, 
                               nodeID, instances.toString() + " instances", null);
        }
        // now create pins and edges
        eNodes = new ArrayList<String>();
        for (PrologClause clause : clauses) {
            nodeID = makeNodeID(clause);
            embedded = clause.getBody();
            for (PrologClause e : embedded) {
                eNodeID = makeNodeID(e);
                newPinID = "pin" + VPrologGraphScene.pinID++;
                createPin(this, nodeID, newPinID, IMAGE_ITEM, 
                        eNodeID, "Embedded");
                existingClause = PrologClause.findClause(clauses, e);
                if (existingClause == null) {
                    if (!eNodes.contains(eNodeID)) {
                    // externally defined (?)
                        createNode(this, randXPoint(), randYPoint(), IMAGE_NODE, 
			    eNodeID, "External", Arrays.asList (IMAGE_EXTERNAL));
                        eNodes.add(eNodeID);
                    }
                }
                createEdge(this, newPinID, eNodeID);
            }
        }
    }

    private String makeNodeID(PrologClause clause) {
        String name;
        Integer arity;
        
        name = clause.getName();
        arity = clause.getArity();
        return name + "/" + arity.toString();
    }
    
    private int randXPoint() {
        return (int) (Math.random() * 800);
    }
    
    private int randYPoint() {
        return (int) (Math.random() * 800);
    }


The new constructor grabs the list of PrologClause (predicate) objects and constructs pretty little images on our diagram using the same createNode etc. methods as before. For the entire program, see the archive files at the bottom of the entry. Fix imports if necessary and save and close the file.


Next, we have to change VPrologTopComponent to call the new constructor with a passed DataObject. Open it and add the following method to the class: 

    public void loadProlog(DataObject dObj) {
        scene = new VPrologGraphScene(dObj);
        jPanel1.removeAll();
        JComponent component =  scene.createView();
        jPanel1.add(component);
    }


Notice that we still have the VPrologGraphScene constructor without arguments called in the top component constructor. Our old friend VPrologAction will still call that, so we can test the functionality of the modeling without invoking PrologAST if necessary.


Now we need to edit ShowDiagram, so open it up and remove the slashes on the win.loadProlog line to uncomment it. Now we should be ready to rock and roll.


Testing The Prolog Model


Now you should clean and build the main project then install it in the target IDE and open the BraveNewWorld project. Open up our old buddy fib1.pro and then go to the menu and select "View" "Show Prolog Diagram". You should see something like this.

Woohoo! This is correct. Unfortunately it isn't very interesting. We need a more complicated Prolog program. Create a new Prolog file (!) called sieve.pro in your project and replace the sample contents with this, a classic Prolog implementation of the Sieve of Erastosthenes: 

/* 
    Sieve of Erastosthenes

    primes(Limit, Ps) instantiates a list Ps containing all 
        primes between 1 and Limit

    from Clocksin and Mellish, "Programming in Prolog"
	
*/

primes(Limit, Ps) :-
	integers( 2, Limit, Is),
	sift( Is, Ps ).

integers(Low, High, [Low|Rest]) :-
	Low =< High,
	!,
	M is Low+1,
	integers(M, High, Rest).
integers(_,_,[]).

sift([], []).
sift([I|Is], [I|Ps]) :-
	remove(I,Is,New),
	sift( New, Ps ).

remove(_, [], []).
remove(P, [I|Is], [I|Nis]) :-
   not(0 is I mod P),
   !,
   remove(P, Is, Nis).
remove(P, [I|Is], Nis) :-
	0 is I mod P,
	remove(P, Is, Nis).


Save the file and perform "Show Prolog DIagram" from the menu again. You should get this.

For testing, you can use various public domain Prolog source files available on the Internet. Try "The Public Domain Prolog Library" for starters. When you're done, you can close the target IDE.


Well, we're done with this segment. We now have a Prolog IDE complete with fins and an automatic headlight dimmer. What we need next is in-place compilation so we don't have to continually switch between NetBeans and our Prolog compiler. You got it -- it's coming next. In the meantime, if you want you can alter PrologAST to not print the informative text, and you can delete VPrologAction.java. If you delete VPrologAction, be sure to also delete the actions in layer.xml ("this layer" then delete the instance in "Actions" "Window" and the shadow in "Menu" "Window").


Files From This Entry



Posted by Hulles in NetBeans at 15:44 | Comment (1) | Trackbacks (0)

GeeWhiz Prolog - Part Three - Adding Language Support

Monday, May 19. 2008

NetBeans
This is Part Three, the fourth part of a series of entries describing an implementation of the Prolog language in the NetBeans IDE.
Part Zero is here.
Part One is here.
Part Two is here.


What We'll Be Doing


Last entry we constructed a simple text editor for Prolog source files in the NetBeans IDE. Now we're going to add "language support" for the Prolog editor, which means that the Navigator window works with the Prolog source and the Prolog syntax is colored and highlighted in our editor. Sounds simple, yes? Remember what Wittgenstein said: "Philosophische Probleme entstehen, wenn die Sprache feiert." ("Philosophical problems arise when language takes a vacation," loosely translated.)


Once again, the procedures in this entry are mostly derived from two NetBeans tutorials, the NetBeans Platform Schliemann Tutorial and Quick Start: Creating Language Tools in NetBeans IDE, and once again, thanks to both authors for their articles.


Add Language Support


The first thing we're going to do is add the language support framework to our project. Right-click on "GeeWhiz" in the project tab and select "Properties."

In the Project Properties window, select "Libraries" under Categories.

To the right of the Module Dependencies section, click the "Add" button.

In the Add Module Dependency window click "Generic Languages Framework," then click "OK". Your project properties should now look like this.

Click "OK" to close the window. The GLF library is now included in our project.



Adapt Language Support to Prolog


Now we're going to create language support for Prolog using our new framework. Right-click "GeeWhiz" in the Project tab and select "New" "Language Support".

In the New File dialog, we're going to reenter the MIME type information for Prolog files in the same way we did when we created the file type last time.

In "Mime Type" type "text/x-prolog" and in "Extensions" type ".pro". Click "Finish" and the wizard will create a couple new files in our project. In the editor window, close the mime resolver xml document but leave the language.nbs file open. ("If you shoot a mime, should you use a silencer?" - Steven Wright) The language.nbs file contains a definition of the grammar for the language we want to support -- in our case, Prolog. The default language.nbs file that is created has a grammar for a simple language already in it but we're not going to use that one. Select the entire contents of the language.nbs file and replace it with this:


########################
# Definition of tokens 
#   thanks to "NetBeans Platform Schliemann Tutorial" author for initial syntax
########################

# Keywords 
TOKEN:keyword:(
    "true" | "fail" | "!" | "at_end_of_stream" |
    "nl" | "repeat" | "halt" | "is" | "mod" | "rem"
)

TOKEN:special:( "." | ":-" | ",")


# Predefined predicates
TOKEN:function:(
    "abolish" | "abort" | "absolute_file_name" |
    "absolute_file_name" | "access_file" | 
    "append" | "append" | "apply" | "apropos" |
    "arg" | "arithmetic_function" | "assert" |
    "assert" | "asserta" | "asserta" | "assertz" |
    "assertz" | "at_halt" | "at_initialization" | 
    "atom" | "atom_char" | "atom_chars" | "atom_length" |
    "atom_to_term" | "atomic" |
    "autoload" | "bagof" | "between" | "block" |
    "break" | "call" | "call" | "call_dll_function" | 
    "call_shared_object_function" | "character_count" |
    "chdir" | "checklist" | "clause" | "clause" | 
    "clause_property" | "close" | "close_dde_conversation" |
    "close_dll" | "close_shared_object" | 
    "compare" | "compiling" | "compound" | "concat" |
    "concat_atom" | "consult" | "context_module" | 
    "convert_time" | "copy_term" | "current_arithmetic_function" |
    "current_atom" | "current_flag" | 
    "current_foreign_library" | "current_functor" | "current_input" |
    "current_key" | "current_module" | 
    "current_module" | "current_op" | "current_output" |
    "current_predicate" | "current_stream" |
    "dde_current_connection" | "dde_current_service" | "dde_execute" |
    "dde_register_service" | 
    "dde_request" | "dde_unregister_service" | "debug" |
    "debugging" | "default_module" | "delete" | 
    "delete_file" | "discontiguous" | "display" | "displayq" |
    "dwim_match" |
    "dwim_predicate" | "dynamic" | "ed" | "ed" | "edit" |
    "edit" | "edit_source" | "ensure_loaded" | "erase" | 
    "exception" | "exists_directory" | "exists_file" | "exit" |
    "expand_file_name" | 
    "expand_file_search_path" | "expand_term" | "explain" |
    "explain" | "export" | "export_list" |
    "feature" | "file_base_name" | "file_directory_name" |
    "file_search_path" | "fileerrors" | 
    "findall" | "flag" | "flatten" | "float" | "flush" |
    "flush_output" | "forall" | "foreign_file" | "format" | 
    "free_variables" | "functor" | "garbage_collect" | "gensym" |
    "get" | "get" | "get0" | "get_single_char" | 
    "get_time" | "getenv" | "ground" | "hash_term" | "help" |
    "history_depth" | "ignore" | "import" | 
    "index" | "initialization" | "int_to_atom" | "integer" |
    "intersection" | "is_absolute_file_name" | 
    "is_list" | "is_set" | "keysort" | "last" | "leash" | "length" |
    "library_directory" | "limit_stack" | 
    "line_count" | "line_position" | "list_to_set" | "listing" |
    "load_foreign" | "load_foreign_library" |
    "make" | "make_fat_filemap" | "make_library_index" | "maplist" |
    "member" | "merge" | "merge_set" | 
    "module" | "module_transparent" | "msort" | "multifile" |
    "name" | "nodebug" | "nonvar" | "noprotocol" | 
    "nospy" | "nospyall" | "not" | "notrace" | "nth0" | "nth1" |
    "nth_clause" | "number" | "number_chars" | 
    "numbervars" | "once" | "op" | "open" | "open_dde_conversation" |
    "open_null_stream" |
    "open_shared_object" | "phrase" | "please" | "plus" | "portray" |
    "portray_clause" | "predicate_property" | 
    "predsort" | "preprocessor" | "print" | "profile" |
    "profile_count" | "profiler" | "prolog" | 
    "prolog_current_frame" | "prolog_frame_attribute" |
    "prolog_load_context" | "prolog_skip_level" | 
    "prolog_to_os_filename" | "prolog_trace_interception" |
    "prompt1" | "prompt" | "proper_list" | "protocol" | 
    "protocola" | "protocolling" | "put" | "qcompile" | "qload" |
    "qsave_program" | "qsave_program" |
    "read" | "read_clause" | "read_history" | "read_link" |
    "read_variables" | "recorda" | "recorded" | "recordz" | 
    "redefine_system_predicate" | "rename_file" | "require" |
    "reset_profiler" | "restore" | "retract" | 
    "retractall" | "reverse" | "same_file" | "save" | "save_program" |
    "save_program" | "see" | "seeing" | "seen" | 
    "select" | "set_feature" | "set_input" | "set_output" | "set_tty" |
    "setarg" | "setenv" | "setof" | "sformat" | 
    "shell" | "show_profile" | "sleep" | "sort" | "source_file" |
    "source_location" |"spy" | "stack_parameter" | 
    "statistics" | "stream_position" | "string" | "string_length" |
    "string_to_atom" | "string_to_list" | 
    "style_check" | "sublist" | "subset" | "substring" | "subtract" |
    "succ" | "swritef" | "tab" | "tell" | "telling" | 
    "term_expansion" | "term_to_atom" | "time" | "time_file" |
    "tmp_file" | "told" | "trace" | "tracing" | 
    "trim_stacks" | "tty_get_capability" | "tty_goto" | "tty_put" |
    "ttyflush" | "union" | "unknown" | 
    "unload_foreign_library" | "unsetenv" | "use_module" | "use_module" |
    "var" | "visible" | "volatile"
    "wait_for_input" | "wildcard_match" | "write" | "write_ln" |
    "writef" | "writeq" |
    "abs" | "acos" | "asin" | "atan" | "atan" | "ceil" | "ceiling" |
    "cos" |
    "cputime" | "e" | "exp" | "float" | "float_fractional_part" |
    "float_integer_part" | 
    "floor" | "integer" | "log" | "log10" | "max" | "min" | "random" |  
    "round" | "truncate" | "pi" | "sign" | "sin" | "sqrt" | "tan" | "xor"
)

TOKEN:string:( "\"" [^ "\""]* "\"" )
TOKEN:string:( "\'" [^ "\'"]* "\'" )
TOKEN:list:( "[" - "]" )
TOKEN:operator: (
    [ "?"  "/" "*"  "-" "+"  "@" "#" "$" "%" "^" "\\"
    ]
)
TOKEN:boolean: (
    [ "<" ">" "=" ","  ";"  "&" "~" "|"
    ]
)

TOKEN:separator: ( ["(" ")" "[" "]" "{" "}"] )

TOKEN:variable:( ["A"-"Z" "_"] ["a"-"z" "A"-"Z" "0"-"9" "_"]* )
TOKEN:identifier:( ["a"-"z"] ["a"-"z" "A"-"Z" "0"-"9" "_"]* )
TOKEN:number:(  ["0"-"9"]+   ("." ["0"-"9"]+)? ("E" ("+" | "-" )? ["0"-"9"]+)? )
TOKEN:whitespace: ( [" " "\t" "\n" "\r"]* )
TOKEN:comment:( "/*" - "*/" )
TOKEN:line_comment:( "%" [^ "\n" "\r"]* ["\n" "\r"]+ )

# Syntax Coloring
COLOR:function: {
    default_coloring:"default";
    font_type:"bold";
}

COLOR:line_comment: {
    default_coloring:"comment";
}

COLOR:list: {
    default_coloring:"string";
}



#####################
# Syntax definition #
#####################

# Comments and whitespaces should be ignored
SKIP:comment
SKIP:line_comment
SKIP:whitespace


# Grammar definition

S = (Statement)*;

Statement = Structure ("." | ":-" ListOfStructures ".");


ListOfStructures = Structure (BooleanOperation Structure)* ;

Structure = Expression |  
            ;

Functor =  | ;

Expression = BaseExpression ((Operation|"is"|"mod"|"rem") Expression)* |
            "(" BaseExpression ((Operation|"is"|"mod"|"rem") Expression)* ")" ;

Operation = ()+;
BooleanOperation = ()+ | "=.." | ","; 

BaseExpression =  | 
                 | 
                 | 
                Function |
                 ;

Function = Functor ["(" ListOfStructures ")"];


# error highlighting
MARK:ERROR: {
    type:"Error";
    message:"Syntax error.";
}

MARK:error: {
    type:"Error";
    message:"Unexpected character.";
}

# brace completion
COMPLETE "{:}"
COMPLETE "(:)"
COMPLETE "[:]"
COMPLETE "\":\""
COMPLETE "\':\'"


# brace matching
BRACE "{:}"
BRACE "(:)"
BRACE "[:]"
BRACE "\":\""
BRACE "\':\'"

# indentation
INDENT ".*(((:-)\\s*)[^.]*)"


# code folding 
FOLD:ListOfStructures: {
    expand_type_action_name:"Expand clause body";
    collapse_type_action_name:"Collapse clause body";
}

FOLD:comment: {
    expand_type_action_name:"Expand Comments";
    collapse_type_action_name:"Collapse Comments";
}


# navigator support
NAVIGATOR:Statement: {
    display_name: "$Structure$";
    icon: org.hulles.geewhiz.PrologNBS.statementIcon;
}


The Prolog syntax stuff we just put into our language.nbs file comes from the Schliemann tutorial, and I will be eternally grateful to the author that I did not have to research Prolog syntax and type out the grammar, particularly the list of built-in functions. See that tutorial for an explanation of what everything does.

For our purposes, we want to go to the bottom of the language.nbs file and change the "icon:" line from "org.hulles..." to "org.myorg...", then save and close the file.


Now we need to create a new Java class called PrologNBS.java. Right-click the "org.myorg.geewhiz" package in our project and select "New" "Java Class".

In the New Java Class window, name the class "PrologNBS".

Make sure the package line is filled in then click the "Finish" button. A new class will be created and opened in the editor. Replace the class definition with the following:


public class PrologNBS {
    
    public static String statementIcon (SyntaxContext context) {
        ASTPath path = context.getASTPath ();
        ASTNode node = (ASTNode) path.getLeaf ();
        node = node.getNode ("ListOfStructures");
        if (node != null) {
            return "/org/netbeans/modules/languages/resources/variable.gif";
        }

        return "/org/netbeans/modules/languages/resources/method.gif";
    }

}


Your class file should now look something like this:

Right-click in the class and select "Fix Imports" and like magic, all the errors go away. I LOVE the "Fix Imports" gizmo! That alone makes me heart NetBeans. Now your class should be fine

and you can close and save it. The PrologNBS.java class that we just created assigns little icons to the Navigator statements, just like in other languages like Java.


Now we are going to add an icon to our editor. Expand "layer.xml" and "this layer" and go to "Editors" "text" "x-prolog" and right-click "language.nbs" and select "Open Layer File(s)".

Select the line that says "file name=language.nbs..." and replace it with this:


                <file name="language.nbs" url="language.nbs">
                    <attr name="icon" 
stringvalue="org/hulles/geewhiz/prolog-star.png"/>
                </file>



Note: you should replace "org/hulles..." with "org/myorg..." in the layer file addition above. (Thanks, Robert.) Your edited layer file should now look like this:



Presto -- our editor has the dreaded Black Star of Prolog associated with it. Save and close the layer.xml file.


Test Language Support


At this point you should clean and build your main project (GeeWhiz) and install it to our target IDE. See the previous entry for instructions on doing this. Once the target IDE is up, open up our target project called BraveNewWorld and wait for all the indexing to die down.


Once all is quiet in the target IDE, select the fib1.pro file in the BraveNewWorld project and open it. You will see (eventually) that the syntax is highlighted and colored in our little Prolog file.

In the Navigator window, notice that there are lines for our Prolog statements in fib1. Experiment with clicking in both the editor window and the Navigator to see how they work. Also, try typing in the editor window and notice how the syntax checking changes and flashes errors, just like it does in Java (except quicker, hopefully).


Before we leave the target IDE, go to the main IDE menu bar and select "Window" "Other" "Token View".

A new tab is created on the left with the tokens resulting from parsing fib1.pro. Clicking on the tokens is coordinated with the editor pane just like it is in the Navigator.

Close that tab then in the toolbar select "Window" "Other" "AST View". This brings up the AST ("Abstract Syntax Tree") tab on the left.

Notice that clicking in AST also is coordinated with the editor window. The AST view is extremely useful for debugging syntax errors resulting from a bad language.nbs file and is handy for some other uses as well, as we will see next time when we build a visual predicate map with the nifty NetBeans Visual Library.


Files From This Entry


Posted by Hulles in NetBeans at 22:32 | Comments (6) | Trackbacks (0)

GeeWhiz Prolog - Part Two - Creating A File Type

Monday, May 19. 2008

NetBeans
This is Part Two, the third part of a series of entries describing an implementation of the Prolog language in the NetBeans IDE. Part Zero is here. Part One is here.


What We'll Be Doing


In this article we will be creating a new NetBeans module for editing Prolog source code files. This module will eventually be part of our NetBeans IDE, just like any of the other modules for supporting languages like C/C++ or JavaScript. The procedures in this entry are mostly a mashup of two NetBeans tutorials, the NetBeans Platform Schliemann Tutorial and Quick Start: Creating Language Tools in NetBeans IDE. My heartfelt thanks go out to both authors for getting me started on this project. The adaptations I've made here are minor, and mostly just involve me getting my thumbprints on the stuff.


When we're done with this entry, we will have a functioning Prolog source code editor, albeit without navigator support and syntax highlighting. We'll add the language support in the next installment. Woohoo! No more Emacs for editing Prolog source!


Create The Project


First we're going to create a new project in NetBeans. If you don't already have a folder where you want to keep your NetBeans projects you should create one. Mine is called Java_Projects, as you will see in the screenshots. I recommend that it not be called NetBeansProjects in your home directory. This is the default project folder name for the NetBeans IDE, and later on we will let NetBeans create that folder where we will be storing projects that we create in the target instance of the IDE.


Start NetBeans and click "File / New Project" on the main menu bar.

In the "New Project" window, select the category "NetBeans Modules" and under Projects select "Module" and hit the "Next" button.


On the next screen, enter "GeeWhiz" for the project name and adjust the project folder as necessary.

Make sure "Standalone Module" and "Set as Main Project" are checked, and hit the "Next" button.


On the next screen, change the code name base to something like "org.myorg.geewhiz". For the module display name, enter "GeeWhiz Prolog".

(In the screenshot I just called it GeeWhiz but later I went back and changed it to GeeWhiz Prolog.) Leave the localizing bundle and XML layer fields alone and click "Finish".


Let NetBeans crank for a while and you should end up with something that looks like this:

Congratulations. You have a new module project. Now let's put stuff in it.


Add A New File Type


The first thing we will be adding to our new project is a new file type that describes our Prolog source code files. NetBeans will use this new file type to grok that a Prolog file you open should be associated with all the cool stuff we'll be adding to the IDE later. So right-click on "GeeWhiz" in the "Project..." tab

and select "New" and "File Type".


On the first screen of the new file type wizard, under "MIME Type" enter "text/x-prolog" and in the "Extensions" field type ".pro".

Technically, Prolog files ought to have ".pl" as a suffix but we don't want to confuse them with Perl files so we'll use ".pro" instead. Hit "Next".


On the next screen of the wizard enter "Prolog" for the "Class Name Prefix".

The wizard will use this as a prefix for all the files it is about to create. For an icon, you can either scrounge up a 16x16 icon on your own system or use the one I used: Right-click the image here and save it on your own computer somewhere, then put the path and file name where you saved it into the wizard. Don't save it in your project folder; the wizard will copy it in there itself.


Now make sure in the "Project" field it says "GeeWhiz" and in the package field it says "org.myorg.geewhiz". Then hit "Finish" and NetBeans will go crazy adding files to your project. When it's done, close all the files (you can right-click the tab of one of them and select "Close all documents") and your project should look something like this.


Edit The Template


One of the files that was just created is called "PrologTemplate.pro". This is the template that will be used to create new Prolog files. Open this file

and replace the contents with the following:


/*
* Prolog Source File ${name}
*   created ${date} at ${time}
*   created by ${user}
*/

% fibonacci(0) = 0
% fibonacci(1) = 1
% fibonacci(2) = 1
% fibonacci(n) = fibonacci(n-1) + fibonacci(n-2)

fib(0,0).
fib(1,1).
fib(2,1).
fib(N,F) :-
    N > 2,
    N1 is N - 1,
    N2 is N - 2,
    fib(N1,F1),
    fib(N2,F2),
    F is F1 + F2.


It should look like this. Save and close the file.

This sample Prolog file is, by the way, slightly adapted from the Schliemann tutorial. I couldn't resist adding the definition of fibonacci(0), which is 0. Also, you can ignore the variables in the comment such as ${name} for now. Later I want to use freemarker to fill these fields in but I haven't figured out how to make it work yet.


Now we're going to add a small HTML file that describes our new Prolog files. Right-click your package ("org.myorg.geewhiz") and select "New" "HTML file".

In the HTML file creation wizard,
enter "PrologTemplate" for the file name. The project field should be filled in already; the folder field should refer to your package ("src/org/myorg/geewhiz"). You should make sure it is filled in in your wizard. It should be there automatically if you right-clicked the package instead of the project, but if not you should select it from the drop-down. Click "Finish" and a new PrologTemplate.html file will be created and opened for you.


In the PrologTemplate.html file,

you should add the following lines between the "body" and "/body" tags:


Create a sample Prolog source file. Prolog is an interpreted language
used for artificial intelligence applications, e.g.


Once you've done that, save and close the file. Now we're going to edit the "layer.xml" file to associate our new description file with the template. we created. In the project window, find "layer.xml" and expand it.

Under "this layer", find "Templates" "Other" "Empty prolog file" and right-click it, then select "Open Layer File(s)". Under the element that says "file name=PrologTemplate.pro..." insert the following line:


                <attr name="templateWizardURL"
urlvalue="nbresloc:/org/hulles/geewhiz/PrologTemplate.html"/>


Change "org/hulles" to "org/myorg" and save and close the file. Next, since the new Prolog file being created from our template isn't really empty, we're going to change the bundle property that describes it. In your project,

expand "Bundle.properties" and "default language" and right-click "Templates/Other/PrologTemplate.pro" and select "Edit". Change the line

for "Templates/Other/PrologTemplate.pro" to say simply "Prolog File", then save and close the "Bundle.properties" file.


Our First Test


Now we're going to test what we've done so far. In the IDE menu, click "Build" "Clean and Build Main Project" and wait until it says "Build Successful" in the output window. Then right-click the GeeWhiz project

and select "Install / Reload in Target Platform". This will install and open a new (target) version of the IDE with our module in it for testing. If you've never, ever, ever made a mistake in your life (hah - we know about that long weekend in Vegas) you can try "Install / Reload in Development IDE", which is much faster since the module is installed right where you're currently working. The disadvantage to this is that if you've screwed anything up, it can trash your development IDE and you'll possibly have to reinstall NetBeans to fix it. I know this from painful experience, so take my advice and always install to the target platform even though it means there will be two instances of the IDE running while you're testing.


While your new target IDE is loading, you'll see all kinds of messages in the devel IDE output box, which you can pretty much ignore unless you experience problems. In the Target, it should eventually say "Finished deploying test module", then "Creating personal domain...".

Wait for it to finish creating your personal domain, whatever the hell that is, then you're ready to test the new stuff.


For starters, create a brand new project in the target. From the target menu, select "File" "New Project" then in the wizard select "Java" "Java Application"

and click "Next". In the next window,

enter "BraveNewWorld" for your project name and let the folder default to the NetBeansProject folder in your home directory. Check "Create main class" and "Set as default project", then click "Finish". The target IDE should now contain the BraveNewWorld Java project. Actually, this project can be any kind of project, we just need one in which we can create new files for now. Later we'll build our own project template for Prolog. By the way, once you add the Java project the target IDE starts indexing all kinds of shit in the background. For performance reasons, you might want to wait until the indexing is done to proceed to the next step.


Now we'll create a new Prolog file in our project. In the target menu bar select "File" "New File..." to start the new file wizard.

In Categories, select Other, then select Prolog file. You should see something resembling the screenshot, with the official GeeWhiz Black Star of Prolog icon next to the Prolog file selection and the description we stuck in PrologTemplate.html in the bottom box of the wizard. Cool, eh?


Now click the "Next" button to travel to the next screen.

For the file name, type "fib1". Make sure the bravenewworld package is selected, then click "Finish". Voilą! Out pops our sample Fibonacci Prolog program, ready to be edited. Give yourself some time to say "Gee whiz! I made that." When you're done, close down the target IDE and return to the devel IDE where we're creating our project, because next we're going to add "language support" to our project so that the navigator works with our file and syntax is (more or less) correctly highlighted. Congratulations.


A Note About The Target IDE


I'm going to take a couple minutes and tell you a little about why I'm doing things the way I am in this how-to series as regards the target IDE. I will be having you clean and build your main project in every case prior to installing to the target IDE. This destroys any target IDE instance you may have created earlier since it cleans the testdir directory. This is a good thing, since in my experience reloading to a target IDE I created earlier almost always causes some sort of weird problem. What this means, though, is that every target IDE starts ab ovo and doesn't remember anything about the last time it was installed. It reminds me of the classic soap opera character with amnesia, to tell you the truth: "Jeez, I didn't know I was married!" That is why we created our target project BraveNewWorld in the default directory, so we don't need to tell the IDE every time where our project directory is located.


And speaking of that, if anyone knows how to tailor the target IDE separately from the devel IDE, please let me know. I have not yet been able to find out where that is documented.


Doh!


In this section, I'll just mention a few of the errors I made while working with the material in this entry, numbered from 0 in true geek fashion.


#0:I originally tried to install to the devel IDE, as I told you not to do above. Doh! It worked fine the first few tests, but then I screwed something up (the layer XML file I think) and crashed NetBeans. I had to reinstall NetBeans and start all over, sadder but wiser.


#1:I had a hell of a time trying to figure out what was up with the layer XML file, based on the tutorials I mentioned. It seems NB 6.1 must be a little different from what the tutorial authors were using, or at least nobody actually said you had to right-click the element of interest and select "Open Layer File(s)". I kept trying to just plain open the element and all I got were property sheets that resembled in no way what the authors were talking about. Doh! Since I'm mentioning it, under "layer.xml" in the project you'll see "this layer" and "this layer in context". "This layer" refers to the layer.xml elements that are actually being modified by our project. "This layer in context" refers to all IDE layer.xml elements, with the ones modified by our project in bold type. This is very useful as you get more familiar with the layer.xml stuff. I am still learning a lot about it.


#2:When I first tested the above stuff, I couldn't for the life of me figure out why I couldn't get a new file description in the new file wizard from PrologTemplate.html. It turned out I created the file in the source directory of the project, not in the "org.hulles.geewhiz" package, so it wasn't being found. Doh! That's why I emphasized that the package should be shown in the Create New HTML File wizard earlier.


Files From This Entry


Posted by Hulles in NetBeans at 14:02 | Comments (0) | Trackbacks (0)

GeeWhiz Prolog - Part One - Before We Start

Sunday, May 18. 2008

NetBeans
This is Part One, the second part of a series of entries describing an implementation of the Prolog language in the NetBeans IDE. Part Zero is here.


Set Up Your Development Environment


The first thing you should do if you haven't already is install NetBeans 6.1 IDE. Most of the stuff here will work with older versions of NetBeans, but much of it will look differently and some of it will function differently, so bite the bullet and upgrade if you haven't already. Once you have it installed, you should fire it up and install the latest upgrades to the plugins. NetBeans should tell you that there are updates available and will install them if you tell it to. It would also be a good idea to play around with it a little bit just to see how it works and maybe create a simple Java application or two. The world is waiting for you to say hello.


Next you should install SWI Prolog on your machine. This step is optional, but if you want to follow along with the last part of this adventure, you'll need it. If you're running Debian or Ubuntu Linux, SWI Prolog is available in the repositories, otherwise you can download it from the SWI Prolog site. You might also play with this a bit. If you're new to SWI Prolog, you can start it from the Linux command line by typing "swipl". You can exit SWI Prolog by typing Ctrl-C then e. That tip right there just saved you about a half-hour of research, by the way.


Pare Down The NetBeans IDE


Now you should make your installed NetBeans IDE leaner and meaner. Uninstall as many plugins from your version of NetBeans as you feel comfortable with, then uninstall a couple more. Pare it down as much as you possibly can, because for much of this how-to you will be running not one, but two instances of the IDE. We will be running both a development instance and a target instance of NetBeans as we are testing GeeWhiz, and unless you're developing on a Cray supercomputer you'll want to speed up the loading and unloading of the IDE as much as possible. Otherwise you'll end up eating many snacks while you're waiting for the IDE to start and put on unwanted pounds and actually start responding to the quick-weight-loss spam email you keep getting, and we don't want that, do we? No we bloody well don't.


Check Out The Giants


As I said in Part Zero, I stand on the shoulders of giants. If you want to check out the big guys, first go to the NetBeans Platform Learning Trail. This page is a good starting point for learning how to extend the NetBeans IDE, which is what we'll be doing. Just FYI, so-called "Rich Client Programming" applications are ones that use the NetBeans platform for a user interface on top of a stand-alone system that may or may not have anything to do with programming languages. We are not building a "Rich Client Programming" application in this how-to. Our application, GeeWhiz Prolog, will be incorporated into the IDE itself so we can use all the goodies that are laying around and switch back and forth between Prolog and other kinds of projects without restarting the IDE. See snack comment, above.


This project initially began when I ran across an article called NetBeans Platform Schliemann Tutorial while searching for an existing Prolog IDE in NetBeans. In this tutorial the author laid much of the groundwork for GeeWhiz and provided the starting point for this project. You might want to hold off actually doing the tutorial, however, as we will be incorporating a modified version of much of the content into GeeWhiz.


Another tutorial to take a look at is the article Quick Start: Creating Language Tools in NetBeans IDE. This is what I used to get my feet wet with adding a language to the NetBeans IDE. It's good and it's quick. Check it out, and go ahead and do it if you want.


Yet another tutorial to which I owe a big debt of gratitude is the excellent A Visual Database Explorer For NetBeans. Take a look at it, but you might want to hold off on doing this one as well since we'll be hacking it later. You'll like it when we do -- "Gee whiz!"

Posted by Hulles in NetBeans at 08:17 | Comments (0) | Trackbacks (0)

GeeWhiz Prolog - Part Zero - About the Project

Saturday, May 17. 2008

NetBeans

Welcome


This is the first entry in a (planned) series of blog posts about creating an integrated development environment (IDE) for Prolog programming using the NetBeans IDE. The purpose of these blog entries is primarily to show how to create a functional editor, diagrammer and compiler wrapper for the Prolog programming language, and secondarily to explore some of the very cool features of the NetBeans IDE.


What's Prolog? What's NetBeans?


Right off the bat, I want to define my terms so you know if you ended up in the right place via your search engine:


Download NetBeans! NetBeans is a free, open-source IDE that does tons of cool stuff. It is for programmers. If you're not a programmer, you should probably quit reading this blog entry right now because you won't find it very interesting. If you are a programmer, you should consider using NetBeans, particularly if you are a Java coder or currently use Eclipse. NetBeans rocks.


Prolog is a programming language commonly used in artificial intellligence applications, natural language processing and expert systems. This blog won't really be addressing Prolog as a programming language per se, although I'll be talking about the language in passing as it pertains to the implementation of the IDE. The Prolog implementation I'll be using for this project is SWI Prolog, a free product that I respect tremendously. If you're going to work along with me as I build the GeeWhiz IDE, you might consider downloading it, although this is not a requirement.


Who Am Us, Anyway?


I should at this point confess that I'm not an expert with the NetBeans IDE, nor am I an expert Prolog programmer. I used to do a great deal of programming in Prolog, back in the days when Borland released Turbo Prolog, a successor to their very popular Turbo Pascal product. Ah, those were the days. Now I've forgotten most of what I knew about Prolog; this project resulted in part from my desire to reacquaint myself with the language. It also resulted from a desire to experiment with the new NetBeans version 6.1. Some very interesting things resulted from this experimentation, as you will see.


Why Prolog?


One of the nice things about using Prolog as a language around which to build an IDE is that the syntax is pretty simple and straightforward, relatively speaking. In absolute terms, however, parsing the language can present a few challenges, which we will gracefully surmount in later installments. Actually, the surmounting wasn't graceful at all, to tell you the truth. Much hacking and swearing and throwing of hamsters went on while I tried to figure the stuff out, but when I present it here, it should look as if I knew what the hell I was doing all along. Heh heh.


Is This A Tutorial? Can I Follow Along?


Hell yes, you should follow along, but this blog series won't really be a tutorial on NetBeans language implementation because I want this to be accessible to the experienced programmer who isn't all that familiar with the NetBeans IDE. I have discovered that much of the official NetBeans documentation assumes that you already know what the hell is going on under the hood. I certainly didn't know much about how the IDE works, at least when I started this project, so I thought I'd pass along what I learned as a help to others in my position. To that end, I will be discussing my own experiences more than should really be the case in a tutorial, including revealing (some of) the many mistakes I made in hacking this creature called GeeWhiz Prolog.


And yes, hacking is the right verb. I haven't studied (for example) the Visual stuff in NetBeans any more than I need to get GeeWhiz up and running. In other words, I know just enough to be dangerous. But if I had tried to become an expert, I'd still be studying months from now, and I have real work to do in the meantime. So I stand on the shoulders of giants (see tutorial references later) and only occasionally jump up and down on their heads. Even so, I think GeeWhiz came out pretty well. For hacking.


Isn't This Just A Rehash Of Other Tutorials?


I suppose in a way this is just a mashup of other information that's already out there. However, what I am bringing to the table is that I put all the information you need to start implementing a new language in NetBeans in one place. That be here, baby. If you don't believe me, do what I did and search NetBeans.org (and the rest of the Internet) for instructions on language support in NetBeans. Most of the pieces are there, but I haven't found anyplace where they are all assembled and integrated, especially in the form of the step-by-step how-to (or, sometimes, how-not-to) that I have tried to create here. You're welcome. See "donate" button.


What Do I Need To Create GeeWhiz Prolog Myself?


You should have an installed version of NetBeans 6.1, which you can get by following the link on the graphic above. You don't absolutely need SWI Prolog installed, but you should consider it as I mentioned above. You should also have had some experience in Java programming. While I am going to assume you know little about the NetBeans platform, I am going to bypass explaining a lot of the Java code I include. This is mostly because I imagine that you probably know more about Java than I do. I consider myself an intermediate-level Java programmer on a good day, and I haven't been having so many good days lately.


I Run On A Linux/Mac/(shudder)Windows Platform. Can I Construct GeeWhiz Prolog?


The platform I've been using to develop GeeWhiz is a creaky old Ubuntu Linux machine code-named Aspen. However, I'm happy to report that both NetBeans and SWI Prolog are multi-platform products. Windows users should be able to run the stuff you see in this GeeWhiz blog series just fine, and Mac users will probably be okay too. You should let me know if that isn't the case, just for future reference.


I Have A Comment...


Speaking of letting me know, I hope you'll pass along any comments you may have on the project. In fact, I'll be including a bunch of questions I have myself as we go, which perhaps some of you can answer.


What Features Does GeeWhiz Prolog Have?


Good question. Glad you're paying attention. When you're done with this series, you will have implemented an environment within NetBeans that includes:

  • a text editor for Prolog source files that includes syntax highlighting and a sidebar navigator
  • the ability to generate a visual and interactive map of Prolog predicate use
  • in-place compilation of Prolog source (using SWI Prolog) with context-linked error and warning messages
  • the ability to easily customize the IDE for use with other Prolog engines
  • a framework upon which other language implementations can be constructed

Of cours