Jibble

Author Topic: MODULE: Modox - Module Documentation Extractor  (Read 2981 times)

RickJ

  • fix'n one thing and break'n two ...
    • I can help with scripting
    • I can help with story design
    • RickJ worked on one or more games that was nominated for an AGS Award!
MODULE: Modox - Module Documentation Extractor
« on: 08 Dec 2009, 04:26 »
This module provides functions that extract comments and code elements from an Ags Script Module's source files and renders an APIi document in one of several formats such as HTML or plain text. The module's script and header files are prepared following a few simple conventions that are easy to remember and apply. The result is natural, attractive, and professional looking source code that can automatically be converted to an API document.

The module is in a pre-alpha state at the moment.  It's basic functions are complete but lacks some critical features such as a user interface, file handling/selection abilities, and currently only renders to an HTML document.  

Here are the module's source files:
     http://demo.agspace.ws/project/modules/Modox/Modox.asc
     http://demo.agspace.ws/project/modules/Modox/Modox.ash

Here are the template files that were used to generate the document:
     http://demo.agspace.ws/project/modules/Modox/ModoxHtml-V0000.html
     http://demo.agspace.ws/project/modules/Modox/ModoxHtml-V0000.css

Here is the html document the module generated from it's own source files:
     http://demo.agspace.ws/project/modules/Modox/Modox.html

Please let me know if anyone has any interest in using this or feedback about the appearance of the output  document.  I am particularly interested to know if the HTML is sufficiently flexible to allow for future and more sophisticated CSS styling to be applied.

The html document renders perfectly in Firefox 3.5.  I am fairly new to CSS and from what I have read there are or have been a number of non-compliant browsers and versions of browsers. So not being an expert I can't guarantee how my CSS styling will look in other browsers. but I am willing to suggestions and feedback.  

Again this is a work in progress and it is not nearly finished.  I am posting at this early stage so that any input people wish to make can receive due consideration. Cheers
Rick

[edit]
Here is a demo project for anyone who would like to try it out.  Click the Read button to read a module doc into memory.  Click the render button to generate the document.   The name of the module can be change by editing the button handler near the end of the Room-3 script file.  Currently the module.asc and module.ash files need to be in the main game directory for this to work.

http://demo.agspace.ws/project/archive/WidgetModule-S0001-000.zip
« Last Edit: 08 Dec 2009, 09:40 by RickJ »

abstauber

  • Cavefish
  • Mittens Knight
  • still mowing the lawn
    • abstauber worked on one or more games that won an AGS Award!
    •  
    • abstauber worked on one or more games that was nominated for an AGS Award!
Re: MODULE: Modox - Module Documentation Extractor
« Reply #1 on: 08 Dec 2009, 08:07 »
Hi Rick,
this is some neat stuff. As I'm used to this kind of tools (javadoc and phpDocumentor), I'm pretty happy to see a doc tool for AGS.
Of course the extra work only makes sense for bigger modules.

I'll try to switch my custom dialog module to your doc style and see what happens :)

RickJ

  • fix'n one thing and break'n two ...
    • I can help with scripting
    • I can help with story design
    • RickJ worked on one or more games that was nominated for an AGS Award!
Re: MODULE: Modox - Module Documentation Extractor
« Reply #2 on: 08 Dec 2009, 09:47 »
Hi abstauber,

If your going to give it a try here is the demo project.  It's hosted in my little sandbox app called BluBox.  BluBox and the Gui support module Widget are also under heavy development at the moment so be aware there are a number of things hanging in an incomplete state.   However, you are welcome to use BluBox in it's current state if you find it useful.  Anyway here is the link:

http://demo.agspace.ws/project/archive/WidgetModule-S0001-000.zip

I would be interested to see what your current doc style looks like.  If you have a convenient example send me a link to it and I'll take a look.

monkey0506

  • SEND PIZZA.
    • Best Innovation Award Winner 2017, for his work to help AGS games reach the widest possible audience - through popular distribution platforms (Steam, Galaxy) as well as other operating systems (Android, Linux)
    • monkey0506 worked on one or more games that won an AGS Award!
    •  
    • monkey0506 worked on one or more games that was nominated for an AGS Award!
Re: MODULE: Modox - Module Documentation Extractor
« Reply #3 on: 08 Dec 2009, 18:10 »
Hey Rick,

This is really keen looking. It's awesome that you came up with the idea/way to do this and the HTML/CSS looks very professional and nice. I'd definitely like to give it a shot myself as well.

Oh and regarding the idea of exporting plain-text docs instead of HTML...could that be done using simply a plain-text non-HTML template? Or does the parsing of the template expect valid HTML?

RickJ

  • fix'n one thing and break'n two ...
    • I can help with scripting
    • I can help with story design
    • RickJ worked on one or more games that was nominated for an AGS Award!
Re: MODULE: Modox - Module Documentation Extractor
« Reply #4 on: 11 Dec 2009, 03:40 »
Quote
This is really keen looking. It's awesome that you came up with the idea/way to do this and the HTML/CSS looks very professional and nice. I'd definitely like to give it a shot myself as well.
monkey, thanks for your kind words.  TerranRich got  me started with CSS and I am grateful for his help.

Quote
Oh and regarding the idea of exporting plain-text docs instead of HTML...could that be done using simply a plain-text non-HTML template? Or does the parsing of the template expect valid HTML?
That's pretty much what I am aiming for.  The rendering process doesn't really care what's in the template file except for the #{...} things.  You can think of the things inside the braces as a command or function with optoinal arguments.  The  command, arguments, and the braces are replaced by whatever text the command produces.  It could be a short string or multiple lines of text. 

   #{command arg1 arg2 ... agr9}

In it's simplest form the #{} just contains a variable name, in which case it is simple replaced by the value of the variable.

I am currently working on an upgraded version of the renderer  where #{HtmlContent} and #{HtmlToc} will be deprecated in favor of a more general purpose ability to define a content model in the template file.   It will have a form similar to this:

Code: [Select]
#{ForEach

   Section {
      <h#{Level}>#{Name}</h#{Level}>
      <p class="definition">#{Definituion}</p>
   }

   Paragraph {
     <p>
     #{Lines}
    </p>
   }

   :
}

The ForEach command makes it iterate over each item that has been extracted from beginning of document to end.  For each item the render will search for a content model that matches the item type which it uses to render the output.  If it doesn't find a content model for that type of item then it moves on toi the next one.  So for example if you wanted to make an index you would just provide a content model for sections. 

Anyway doing it this way I will be able to render to a wider variety of different formats just by providing a different template file.   Anyway I'm about half way into the version of the renderer.  I am currently putting things all back together after having broken the whole thing. ;)  Not to worry I have an archive of the previous version.

I'll be back when I have a new version..


monkey0506

  • SEND PIZZA.
    • Best Innovation Award Winner 2017, for his work to help AGS games reach the widest possible audience - through popular distribution platforms (Steam, Galaxy) as well as other operating systems (Android, Linux)
    • monkey0506 worked on one or more games that won an AGS Award!
    •  
    • monkey0506 worked on one or more games that was nominated for an AGS Award!
Re: MODULE: Modox - Module Documentation Extractor
« Reply #5 on: 15 Dec 2009, 16:38 »
I've just noticed that when the functions are being extracted the documentation includes the "import" keyword. This seems a bit superfluous. Anyway, I'm working on typing up some documentation in your specified format to see what kind of results I can get with your module. :)

If I have any other concerns or thoughts I'll let you know.

Edit: Okay I've encountered a few more things that I'd like to see addressed:

-writeprotected members are ignored entirely. The module currently does not detect these properties at all, though since they are public they should be documented.

-Member descriptions are omitted. I was able to work around this by adding on a line by itself something such as "// blarg:" at the start of each property description.

-Pointer members...are strange. I found that if you declare your pointers such as Type* Name (with the asterisk attached to the type) that the member is ignored entirely, yet if you declare it as Type *Name or even Type * Name then the asterisk is considered part of the property name!

-Version macros required odd formatting. It seems that in order to get the version macros to detect properly they must be padded to 4 spaces. This is far from desirable for me. As a matter of fact, I would like to be able to provide the VERSION macro as a float.

-No distinction between global and struct functions? I thought this was already implemented, yet when I run my module through yours the output does not reflect this. The struct name is reflected, its members listed, and then a global instance of that struct, as well as a global function are listed, but with nothing indicating they are not part of the struct. Actually as a matter of fact I would like to be able to avoid referencing the struct name altogether and only mention the instance because that is the user access interface.

I think that's all I can really comment on for now. Still a nice piece of work, but it requires some tweaking. ;)
« Last Edit: 16 Dec 2009, 00:05 by monkey_05_06 »

RickJ

  • fix'n one thing and break'n two ...
    • I can help with scripting
    • I can help with story design
    • RickJ worked on one or more games that was nominated for an AGS Award!
Re: MODULE: Modox - Module Documentation Extractor
« Reply #6 on: 16 Dec 2009, 15:35 »
Monkey, thanks for the great feedback.  I am putting the finishing touches on a re-factored renderer. I have to do some code clean-up and update the html template file so that the same document will be produced as before.  The new renderer doesn't know or care what is being rendered.  All HTML tags are now contained in the template file.  

Html TagsIn it's new incarnation text files are supported the same as html.  It should also be possible to create template files that will produce

All tags and other markup are now specified in the template file and are no longer part of the Modox script.

Plain Text Output
Plain text files can now be rendered, however,  support for page break, margins,  and
line fill, line justify, wrap, etc needs to be added

Multiple Output FilesThe output file is now specified inside the template file.  The file name may be constructed using variable substitution so it's possible to use things like module name and version to construct the file name.  It's also now possible to produce multiple output files from one template.

CHM Help
It ought to be possible to render the required file set to create CHM help files.  If anyone has experience with this I could use some help or advice in this area.

I am also planning a review of the parser so your feedback comes at an opportune time.  Let me respond briefly to each point you raise.

Quote
I've just noticed that when the functions are being extracted the documentation includes the "import" keyword.
Perhaps so.  I just picked up whatever was on the line and only excluded the punctuation characters  ";" and "{" at the end of the line(s).  I can exclude the import statements if thats the consensus of how it ought to be.  I don't really have a preference one way or the other, just what is best way for the most people.

Quote
-writeprotected members are ignored entirely. The module currently does not detect these properties at all, though since they are public they should be documented.
I was wondering about that and you know I didn't have any examples and wasn't certain public properties could be write protected.  What is the proper form anyway/
Code: [Select]
    protected int SomeVariable;

Quote
-Member descriptions are omitted. I was able to work around this by adding on a line by itself something such as "// blarg:" at the start of each property description.  
The parser is expecting the description to follow the declaration.   The  declaration  statements
serve as both definition of a software entity and a section title for the document.  The following ought to be picked up by the parser, if not then there is a bug.  Also note that in the case of object methods, global functions and global variables content will also be picked up from the script file and will also be included.
Code: [Select]
    int SomeVariable;
     // This variable is used for this that and the other thing.
     //  The description of the variable follows the declaration so
     // so these lines are the description of SomeVariable.

Quote
-Pointer members...are strange. I found that if you declare your pointers such as Type* Name (with the asterisk attached to the type) that the member is ignored entirely, yet if you declare it as Type *Name or even Type * Name then the asterisk is considered part of the property name!
Ahhh ok!  I understand what's going on here.  I keep a list of all the built-in AGS types and add to that list each of the user defined types as they are defined.  So when the parser sees something like "int",  "Monkeys_Object", etc it knows that this line needs further analysis.  This should be fixed in the next version.

Quote
-Version macros required odd formatting. It seems that in order to get the version macros to detect properly they must be padded to 4 spaces. This is far from desirable for me. As a matter of fact, I would like to be able to provide the VERSION macro as a float.
I got this straight out of the module guidelines.  Perhaps I am a bit out of date or this is a bit too strict.   post some examples and I'll see what I can do about getting them be recognised.

Quote
-No distinction between global and struct functions? I thought this was already implemented, yet when I run my module through yours the output does not reflect this. The struct name is reflected, its members listed, and then a global instance of that struct, as well as a global function are listed, but with nothing indicating they are not part of the struct. Actually as a matter of fact I would like to be able to avoid referencing the struct name altogether and only mention the instance because that is the user access interface.
Is it the case that the global instance import statements follow the struct definition?   If so what you are saying is that since the methods inside the struct and the global functions both have the same header level there doesn't appear to be a distinction between them when they are rendered into a document?  Have I understood this correctly?  

Internally the parser knows that the struct definition has ended and it categorises the imported global instance as a global function.   I'm not sure what ought to be done here.   Functions, variables, and constants are setup as header level 2 entities.  This presumes they are proceeded by a header level 1 document section.  In the case of methods and properties the struct definition serves this purpose.   There are currently a couple of options and a few more with the new renderer.

Simple Solution
The simplest thing to do is either create a header level 1 section just before the global instance import statement or move it so that it follows an appropriate level 1 section.

Alternative Solution
The new renderer allows specific types of content to be rendered (or not rendered) as the case may be.  So for example, in HTML form, the struct and it's properties and methods could be rendered with html id attribute id="MyStructName"  and then the styling could be adjusted to make anything with that id be invisible.  The template could also be designed to change the grouping of different kinds of items or not render them at all

Maybe we can talk about this more when I get the next version out, since it may already have the required solution implemented.  


monkey0506

  • SEND PIZZA.
    • Best Innovation Award Winner 2017, for his work to help AGS games reach the widest possible audience - through popular distribution platforms (Steam, Galaxy) as well as other operating systems (Android, Linux)
    • monkey0506 worked on one or more games that won an AGS Award!
    •  
    • monkey0506 worked on one or more games that was nominated for an AGS Award!
Re: MODULE: Modox - Module Documentation Extractor
« Reply #7 on: 16 Dec 2009, 16:14 »
Monkey, thanks for the great feedback.  I am putting the finishing touches on a re-factored renderer. I have to do some code clean-up and update the html template file so that the same document will be produced as before.  The new renderer doesn't know or care what is being rendered.  All HTML tags are now contained in the template file.

You're very much welcome and I'm glad to hear that the HTML tags are now exclusively contained in the template file. Does this mean that the template formatting can be preserved too? What I mean is that every single tag in the current output is left-aligned which makes the source an unreadable mess. I've been working to hand-edit the file to account for other issues as well, but it would have been much faster with the template formatting in-tact (as I did edit the indentation and such on the template file).

CHM Help
It ought to be possible to render the required file set to create CHM help files.  If anyone has experience with this I could use some help or advice in this area.

I'm actually not sure whether CHM can be rendered using plain-text. Seeing as the C stands for "Compiled" I think it has to explicitly be passed through the CHM compiler to generate a valid CHM file. I could very well be wrong though. :)

Quote
-writeprotected members are ignored entirely. The module currently does not detect these properties at all, though since they are public they should be documented.

I was wondering about that and you know I didn't have any examples and wasn't certain public properties could be write protected.  What is the proper form anyway/
Code: [Select]
    protected int SomeVariable;

AGS supports protected and writeprotected members. The proper ordering of things is:

Code: [Select]
struct MyStruct {
  protected int ProtectedData; // this is essentially the same as "private" in C++
  writeprotected int WriteProtectedData; // this can be publicly read, but can only be set privately
  protected import function ProtectedFunction(); // this is just an example of a protected function
};

Protected members probably don't need to be documented because they're not intended as part of the user interface. However, writeprotected members are publicly readable and so should be documented. ;)

Quote
-Member descriptions are omitted. I was able to work around this by adding on a line by itself something such as "// blarg:" at the start of each property description.  
The parser is expecting the description to follow the declaration.   The  declaration  statements
serve as both definition of a software entity and a section title for the document.  The following ought to be picked up by the parser, if not then there is a bug.  Also note that in the case of object methods, global functions and global variables content will also be picked up from the script file and will also be included.
Code: [Select]
    int SomeVariable;
     // This variable is used for this that and the other thing.
     //  The description of the variable follows the declaration so
     // so these lines are the description of SomeVariable.

This is the exact code from one of the members of my struct:

Code: [Select]
  ///CountDown module: Gets/sets whether the countdown should be paused when the game is paused.
  bool PauseWhileGamePaused;
  // blargh:
  // Gets/sets whether the countdown should be paused when the game is paused. If the
  // countdown was already paused it may become unpaused in the event that this
  // property is set to TRUE and the game becomes unpaused.
  //
  //   See also: PauseGame, UnPauseGame, IsGamePaused, CountDown.Paused

If I remove the line "  // blargh:" then the description does not show up at all.

Quote
-Pointer members...are strange. I found that if you declare your pointers such as Type* Name (with the asterisk attached to the type) that the member is ignored entirely, yet if you declare it as Type *Name or even Type * Name then the asterisk is considered part of the property name!
Ahhh ok!  I understand what's going on here.  I keep a list of all the built-in AGS types and add to that list each of the user defined types as they are defined.  So when the parser sees something like "int",  "Monkeys_Object", etc it knows that this line needs further analysis.  This should be fixed in the next version.

Great! :)

Quote
-Version macros required odd formatting. It seems that in order to get the version macros to detect properly they must be padded to 4 spaces. This is far from desirable for me. As a matter of fact, I would like to be able to provide the VERSION macro as a float.
I got this straight out of the module guidelines.  Perhaps I am a bit out of date or this is a bit too strict.   post some examples and I'll see what I can do about getting them be recognised.

I don't recall the guidelines saying that the version number should be padded out to exactly 4 digits. In fact, it shows three digits.

I don't have a problem conforming to that standard, but having to define it like:

Code: [Select]
#define Module_VERSION 0100
Is unnecessary.

Quote
-No distinction between global and struct functions? I thought this was already implemented, yet when I run my module through yours the output does not reflect this. The struct name is reflected, its members listed, and then a global instance of that struct, as well as a global function are listed, but with nothing indicating they are not part of the struct. Actually as a matter of fact I would like to be able to avoid referencing the struct name altogether and only mention the instance because that is the user access interface.
Is it the case that the global instance import statements follow the struct definition?   If so what you are saying is that since the methods inside the struct and the global functions both have the same header level there doesn't appear to be a distinction between them when they are rendered into a document?  Have I understood this correctly?  

Internally the parser knows that the struct definition has ended and it categorises the imported global instance as a global function.   I'm not sure what ought to be done here.   Functions, variables, and constants are setup as header level 2 entities.  This presumes they are proceeded by a header level 1 document section.  In the case of methods and properties the struct definition serves this purpose.   There are currently a couple of options and a few more with the new renderer.

Simple Solution
The simplest thing to do is either create a header level 1 section just before the global instance import statement or move it so that it follows an appropriate level 1 section.

Alternative Solution
The new renderer allows specific types of content to be rendered (or not rendered) as the case may be.  So for example, in HTML form, the struct and it's properties and methods could be rendered with html id attribute id="MyStructName"  and then the styling could be adjusted to make anything with that id be invisible.  The template could also be designed to change the grouping of different kinds of items or not render them at all

What I mean is that the TOC is showing:
[h1]StructName[/h1]
[h2]Struct properties...[/h2]
[h2]...[/h2]
[h2]...[/h2]
[h2]Global instance of struct[/h2]
[h2]Global function[/h2]
The code in question goes like this:

Code: [Select]
struct StructName {
  int IntProperty; // documentation lines and such removed for simplicity sake :P, this is all just pseudo code anyway
  String StringProperty;
  // etc.
};

import StructName StructMember;
import function FunctionName();

What should I presumably be putting then between these so that there will be a distinction between the struct's members, the instance, and the global function instead of them all just being on the same header level with nothing separating them?

Anyway like I said, I'm already hand-editing the file anyway, but I like where this is going. :)

RickJ

  • fix'n one thing and break'n two ...
    • I can help with scripting
    • I can help with story design
    • RickJ worked on one or more games that was nominated for an AGS Award!
Re: MODULE: Modox - Module Documentation Extractor
« Reply #8 on: 16 Dec 2009, 20:57 »
Thanks for the feedback. 

Quote
You're very much welcome and I'm glad to hear that the HTML tags are now exclusively contained in the template file. Does this mean that the template formatting can be preserved too? ...
No and yes!  ;)    I am using the same function to read lines out of both the template and source files.  The parser needs to ignore leading white space so the line reading routine does this first thing.   This is also true for some aspects of interpreting the template file.  I'll see what I can do about preserving the horizontal formatting of the template file.

Quote
CHM Help ...
Quote
...  I think it has to explicitly be passed through the CHM compiler to generate a valid CHM file.
There is a set of, I believe three files, that is passed through the CHM compiler which is freely available from M$.  The idea would be to first render the source files and then pass the result through the CHM compiler to get get the final chm.

Quote
writeprotected int WriteProtectedData; // this can be publicly read, but can only be set privately
Thanks.  Leaving this out was an oversight on my part.  I'll get it fixed for the next version.

Quote
If I remove the line "  // blargh:" then the description does not show up at all.
I am certain I had this working correctly at one point in time.  I must have generated a bug along the way somewhere.   I'll get it fixed for the next version.

Quote
Re: versions
...I don't recall the guidelines saying that the version number should be padded out to exactly 4 digits. In fact, it shows three digits. ...

The parsing of the version number can certainly be made more flexible.  For the next version I plan to allow the version number to be of variable length (min of 3 chars) with the last two chars being the minor version.   What other variations should I attempt to accommodate?   

Quote
What should I presumably be putting then between these so that there will be a distinction between the struct's members, the instance, and the global function instead of them all just being on the same header level with nothing separating them?

You could just add a comment line with a level-1 section title as shown in the example below.  You could also, of course, add additional paragraph text and other content as part of the section.
Code: [Select]
struct StructName {
  int IntProperty; // documentation lines and such removed for simplicity sake :P, this is all just pseudo code anyway
  String StringProperty;
  // etc.
};

// 0 StructName Instances
import StructName StructMember;
import function FunctionName();

[h1]StructName[/h1]
[h2]Struct properties...[/h2]
[h2]...[/h2]
[h2]...[/h2]
[h1]StructName Instances[/h1]
[h2]Global instance of struct[/h2]
[h2]Global function[/h2]

monkey0506

  • SEND PIZZA.
    • Best Innovation Award Winner 2017, for his work to help AGS games reach the widest possible audience - through popular distribution platforms (Steam, Galaxy) as well as other operating systems (Android, Linux)
    • monkey0506 worked on one or more games that won an AGS Award!
    •  
    • monkey0506 worked on one or more games that was nominated for an AGS Award!
Re: MODULE: Modox - Module Documentation Extractor
« Reply #9 on: 16 Dec 2009, 22:10 »
Well, in the mean time of waiting for the next version, I've just resorted to completely hand-editing everything. There were also a couple of issues with the generated XHTML not being entirely valid. You can check with the W3C's validator and you should see what I mean. Specifically I recall you are using the name attribute for internal linking, XHTML requires you use the id attribute instead, which is also actually more strict (certain characters such as parenthesis, whitespace, etc. are not accepted).

I think I've actually spent more time editing the documentation on this than I spent writing the module, but I like the layout of it, and it will be fairly easy for me to write a PHP interpreter to convert the HTML into a wiki format. := Of course with this new version of your module it should be possible to just write up a wiki template (text-output for pasting into the wiki). ;)

Well I should actually have my new module up shortly. As you might have guessed if you read over that code snippet I posted of a public property it's a countdown module to simplify the process of...you guessed it...implementing a countdown scenario in your game, along with a couple of options for automatically triggered events. Anyway, I should finish touching up this documentation so I can post the thing already. 8)

RickJ

  • fix'n one thing and break'n two ...
    • I can help with scripting
    • I can help with story design
    • RickJ worked on one or more games that was nominated for an AGS Award!
Re: MODULE: Modox - Module Documentation Extractor
« Reply #10 on: 17 Dec 2009, 06:59 »
I had a look at your new module's documentation and it looks really nice.    I appreciate the effort you are making as it makes me feel like my effort is worthwhile.   I have completed the scripting of the next version and I am currently updating and testing the template and CSS files.  So I should have the new version out later tonight or early tomorrow, depnding on how sleepy I get.

I've addressed the following issues you pointed out :

- writeprotected properties, 
These are now included in the document

- property comments
Comments following property declarations are now included in the document

- version parsing
It is now more sophisticated and flexible. It supplies formatting for 3..N character interger version numbers and version numbers containing a decimal point.  If another format is used it just passes it through unchanged. 

I'll let you know when I have the new version uploaded.  In the meantime here are some things I noticed in your document and some things to think about...

Note:
I'm not sure if you are aware of this feature but if you made your "Note:" sections as preformatted text the resulting html tag would be given a class of "Note" allowing you to style it as you please.
Code: [Select]
<pre class="Note">
Your note text .....
</pre>

Table Of Contents and Other Content Formatting
You will be happy to know that you will be able to accomplish what you have done  by just changing the template.   You will have full control over the header level of functions, methods, properties, etc from within the template file.

See Also:
I noticed and like you "see also" feature.  I'll think on how to do this without manually editing the HTML.

Horizontal Spacing/Format
I'm still thinking on this and have decided to defer action until I begin rendering to plain text.  I think some related issues will arise and so I would like to address this issue at that time.   A concern I have is that on the one hand it may be desirable to use indentation to make the template file easier to read and maintain but it may not be desirable to have that same indentation show up in the output file.   This will be a bit more clear when you see what the new template file looks like.
 
That's all for now.  Need to get back to work and get the next version out...

monkey0506

  • SEND PIZZA.
    • Best Innovation Award Winner 2017, for his work to help AGS games reach the widest possible audience - through popular distribution platforms (Steam, Galaxy) as well as other operating systems (Android, Linux)
    • monkey0506 worked on one or more games that won an AGS Award!
    •  
    • monkey0506 worked on one or more games that was nominated for an AGS Award!
Re: MODULE: Modox - Module Documentation Extractor
« Reply #11 on: 17 Dec 2009, 08:44 »
Regarding the "Note" sections, are you saying that if I were to have done:

Code: [Select]
//   NOTE: blah blah blah
//   blah blah blah

That Modox would have already included it as preformatted text with a CSS class of "Note"? I wasn't aware of that (actually I'm fairly certain it was contained within
 tags, but no class was specified), but I specifically didn't want it to be preformatted text, though with the CSS that can all be managed (font-family, font-size, and white-space I believe are the relevant CSS properties...maybe border, padding, and margin as well). As you can see I just manually added the class to create the specific formatting I wanted. Anyway, thanks for the tip.

Glad to hear that the new version will include more goodies like writeprotected members and improved version parsing.

As for the "see also" sections I think if it were possible it might be beneficial to add some support for linking. I think something that could be given multiple arguments would be great, something like
{link|NAME|PAGE_URI} where PAGE_URI is optional. If only the name is specified it could be assumed that you are linking to the anchor by the same name:

Code: [Select]
{link|CountDown.Hours}
Would generate:

Code: [Select]
<a href = "#CountDown.Hours">CountDown.Hours</a>
If an anchor is specified then the link could be something more like:

Code: [Select]
{link|Hours|#CountDown.Hours}
// ...
<a href = "#CountDown.Hours">Hours</a>

And finally if a PAGE_URI is given then it could produce an external link:

Code: [Select]
{link|PauseGame|http://www.adventuregamestudio.co.uk/manual/PauseGame.htm}
// ...
<a href = "http://www.adventuregamestudio.co.uk/manual/PauseGame.htm">PauseGame</a>

Or:

Code: [Select]
{link|ContainsHowManyOf (String)|http://www.americangirlscouts.org/agswiki/StringPlus#String.ContainsHowManyOf}
// ...
<a href = "http://www.americangirlscouts.org/agswiki/StringPlus#String.ContainsHowManyOf">ContainsHowManyOf (String)</a>

Of course this only makes sense if an HTML template is given...perhaps it could be left up to the template to deal with the specifics like:

Code: [Select]
<!-- HTML template file -->
#{Foreach

  Link {
    <a href = "#{LINK_PAGE_URI}">#{LINK_NAME}</a>
  }

}

Code: [Select]
Plain-text template file:

#{Foreach

  Link {
    Visit #{LINK_PAGE_URI} for more information on #{LINK_NAME}.
  }

}

These are of course just my thoughts on the matter, but if it's feasible to implement something like this then I think it would make it much easier to produce these type of "See also" sections without having to manually edit the output (which of course is ultimately the goal I presume; a perfect document extraction would require no hand-editing at all). (Edit I modified my thoughts on the linking idea somewhat...)

I'm actually not clear though why you think it would be better under any circumstances for the output to not have indentation applied. Compare for example the source of the output file you got (Modox.html) to my hand-edited version (CountDown_Manual.html). In yours everything is left-aligned. There's no indication which tags are children/parents of each other. If a tag (opening or closing) is missing it would be hell to try and manually edit to find where the problem arose from. In my file everything is aligned so at a glance you can easily see the document structure, which tags are ancestors or descendants of which other tags...

Now granted in the world of script-generated source files, it is common for the source to be an unreadable mess like this, so it wouldn't be anything new if the output was void of structure. For example, open up the source for any wiki document anywhere and you'll find that much of it lacks indentation...and where it does have indentation it's totally inconsistent. This has always been a peeve of mine, but I understand the reasoning behind it ultimately. I have written enough scripts in PHP that I know what it looks like trying to produce properly aligned output while preventing the PHP source files from becoming unreadable themselves. It's not easy or fun, and I think that's why people have become apathetic or negligent of the care of the actual output (so long as it looks right when interpreted by the browser).

So if it's too much of a hassle, what I'm getting at, is it wouldn't be the end of the world if you left that particular bit out. I'd much prefer it to be available, but I'll survive if it's not. :)
« Last Edit: 17 Dec 2009, 18:57 by monkey_05_06 »

RickJ

  • fix'n one thing and break'n two ...
    • I can help with scripting
    • I can help with story design
    • RickJ worked on one or more games that was nominated for an AGS Award!
Re: MODULE: Modox - Module Documentation Extractor
« Reply #12 on: 19 Dec 2009, 05:14 »
Since my  last update I've uncovered and corrected a number of bugs that surfaced when imperfect content model was specified.  Anyway that's why I am delayed getting out the next version.   However, the good news is that I've implemented some features that allow the "See Also" and notes sections to be generated without hand coding.

The first thing to note is that in the template file commands and variable substitutions have been separated and now each have a different form.  This, I believe, is a bit clearer and easier to explain and it also makes the underlying implementation less complicated.

Code: [Select]
// Variable have the same form as before.  For example the string  #{SomeVariable} is
// replaced with the value of SomeVariable.
//
// Commands start with a # symbol, kid of like macros in an include file.  They can have
// a number of arguments and span multiple lines using the curly braces  to delimit their extent.
// The for-each command for example has the following form

#for-each {
     // Basic content model
     item-type {
          <tag>#{VariableName}</tag>
     }

     // Model that iterates over each of the item's lines
     item-type-Begin {
          <tag>
     }
     item-type-Lines {
          <tag>#{Line}</tag>
     }
     item-type-End {
          </tag>
     }

     // Model that iterates over each of the item's lines
     item-type-Begin {
          <tag>
     }
     item-type-Lines {
          <tag>#{Line}</tag>
     }
     item-type-End {
          </tag>
     }
}

The two features that make "See Also" possible are Word Iterators and Custom Item Types.  
A custom item type can be created by beginning a paragraph with a set of square brackets enclosing the name of the custom item type.
Code: [Select]
This is normal paragraph text blah, blah, bla bla bla blaaaah.
This is more of the same.   This item will be of type eModox_Paragraph
having a text value of "Paragraph" which can be accessed in the template file
using the variable #{Type}.

[See Also] This is also paragraph text but it has an item type of eModox_Custom
and it has a text value of  "SeeAlso" (note spaces are removed) which is also accessed in the template file via #{Type}.

Now that we have an item type specific to the "See Also" sections we can construct a content model to render the text and html tags in the proper form.   But to do this we need to have a variable for each word so "Words" iterator was added to accommodate this need and the resyulting content model looks like the following.
Code: [Select]
// Commands start with a # symbol, kid of like macros in an include file.  They can have
// a number of arguments and span multiple lines using the curly braces  to delimit their extent.
// The for-each command for example has the following form

#for-each {
     // Model that iterates over each word in the item
     // implements "See Also" feature
     SeeAlso-Begin {
          <p class="#{Type}">#{Name}:  
     }
     SeeAlso-Words {
          <a href="##{Word}" class="#{Type}">#{Word}</a>,
     }
     SeeAlso-End {
          </p>
     }
}

The "Note:" and other similar section formatting can be achieved using either custom item types or the pre-format class feature mentioned in the previous post.  I used the pre-format class feature to implement the example box in my demo document.

The other thing that has changed is that each software entity has it's own item type an so it's render format is under complete control of the template file.  This means that header level and other formatting can be applied depending upon what type of thing it is.;  For example, the navigation scheme in monkey's document (i.e CountDown module) can be easily implemented by making the appropriate template file.

Before I release this version I plan to add a couple more features which should not be very difficult to implement and will hopefully eliminate most of the remaining needs you have (i.e monkey) to hand edit the output file.   This will also give me a little more testing time as well.

Those two features are FullNames for functions, properties and methods and HTML Passthrough.
Previously I had formed the names of methods and properties by pre-pending the object name and a dot as monkey has done in his document.  Somewhere along the line I saw other documentation that didn't do that and I talked myself out of it.   Now after having seen what monkey has done and having thought through the "See Also" feature it is clear that using the current short names  (i.e. names without the object name) to create links will be problematic.   What's needed is a second name variable #{FullName} for each item and that is accessible from the template file.

The second thing to do is to be sure that raw HTML/XML tags appearing in the source document are passed through to the output.  I believe that this is mostly the case at the moment except if the tag is found at the beginning of a line.  So this just needs to be reviewed and minor adjustments made to accommodate this.

Anyway monkey if you can think of anything else that needs to be in here or any other kinds of things requiring hand editing please let me know.  

[edit]
Monkey, I was just thinking about the horizontal alignment issue you mention.   I don't think it's that terribly difficult to preserve the horizontal spacing but I am concerned about that having undesirable affects on the rendering of plain text and possibly other formats.   After I get the next version out I plan to start on adding the things needed to render to plain text.  Things like commands to set page size and margins, function to do line wrap and fill, page breaks and numbering etc.  I'd like the opportunity to consider what to do about horizontal spacing  at that time since I'll have a better idea of what is needed for plain text.

However, in the meantime I just had a bit of a brain storm I'dl ike to share and get your input.  I was thinking "What if the horizontal spacing of the output could be specified in the template file separate from the horizontal spacing of the template file itself?   This would address my concern,  I think it would also meet your needs as well, and would probably be a piece of cake to implement.  Confused yet?  Here's an example of what I am talking about:
Code: [Select]
#for-each {
     // Make level-1 sectionz one all the way to the left margin
     Section.1 {
          <h1>#{Name}</h1>
     }
     // Make level-2 sections indented 5 spaces
     Section.2 {
          .....<h2>#{Name}</h2>
     }
     // Make paragraph tags indented 10 spaces
     // Make paragraph text indented 15 spaces
     paragraph {
          ..........<p>
          ...............#{Lines}
          ..........</p>
     }
     // In the above examples the leading dot characters
     // will be replaced in the output file by space characters.
}
This would allow the template file to be indented in whatever way is needed to make sense of the template file, independently from the indentation desired in the output file.   Anyway it's just something that came to me a few minutes ago and I haven't had time do digest it yet.  Please feel free to opine... ;)
« Last Edit: 19 Dec 2009, 05:58 by RickJ »

eri0o

Re: MODULE: Modox - Module Documentation Extractor
« Reply #13 on: 08 Jan 2020, 18:35 »
I know this is ten years old, but does someone have a backup of this?

Re: MODULE: Modox - Module Documentation Extractor
« Reply #14 on: 09 Jan 2020, 05:44 »
My official site: http://www.pershaland.ir/

eri0o

Re: MODULE: Modox - Module Documentation Extractor
« Reply #15 on: 09 Jan 2020, 09:37 »
Thanks @Mehrdad!!! :D Downloaded! :)