Is it possible to make a module for entering words into the text parser?

[Perpetuall]

So, here I am plugging away at entering long lines of text into my list of text parser words 3 words at a time, when it suddenly dawned on me: Since the AGS engine only allows for up to 30,000 words in the text parser, and I have over 31,000 individual lines of text that I will be eventually entering in, I will inevitably run out of space in the future before I finish completing the series of games I am working on.
Which really zukz.

So my question is two fold.
First, how do I find a way around this limit?

Second, is there a better way to do this? Like, could you in theory make a module for a way to just copy and paste words into the parser a page at a time, and it would just automatically add them all? Because that would save me, like, 80% of my time creating this game.

[Khris]

I made this a while ago:
https://khris.ddns.net/agsparser/

Definitely make a backup of your game before using this though!

Still, I have to wonder why on earth you would need a parser that understands more than 30,000 words.

Just to be clear: you don't need the AGS Parser to check user input. You can just do something like:

Code: ags

  if (password == "1234abc")

So let's make sure this isn't an xy problem first.
Which game mechanic are you trying to implement?

[Perpetuall]

Wow, that is awesome. I'm glad I asked haha. 

My main combat mechanic is quoting lines of Scripture via the text parser from the Bible to exorcise demons. The Bible has 31,000 verses in it. I'm entering each verse as if it were one word.

Would that work for this?

As I'm writing this I'm also running into another problem.

I was testing it in windowed mode as I was working on it just for ease of use, but I just now tried to test it in full screen mode and it won't run. It tells me "Can not set video mode". What's causing this? This is the second time it's suddenly started doing this to me, and I can't figure it out. I keep trying different resolutions but it just won't run in full screen.

Edit:
Come to think of it, it would also really cut my workload if there was a way to just check for certain words in the input. So whenever somebody writes a line that contains a keyword, like "humility", it just automatically triggers the code for that keyword. What's a way to go about doing that? To first, compare what's been inputted to make sure it actually matches the verse and isn't just a bunch of made up stuff, and second, check to see what words are in it and depending on that, determine what to do.

[Khris]

First, the fullscreen thing. What is your game's resolution? What is your desktop resolution? "Cannot set video mode" probably happens if you try to run a 400x400 game on a widescreen monitor, so it really depends on the aspect ratio and the specific winsetup settings you used.

As for detecting bible verses, AGS supports:
a) comparing two string case-insensitive
b) parsing a string using the built-in parser
c) completely custom code

a) is self-explanatory, and the least lenient variant
b) requires adding tons of words to the parser but I don't think you can have words with spaces in them. You'll also basically doing a) because the player isn't typing commands but fixed sentences
c) probably the best approach, but requires careful thought

Let's take "Humble yourselves before the Lord, and he will lift you up.".
What's going to be valid input here? Can there be typos? Missing words?

This is a hard problem, and not one that is solved by using the AGS parser. You can try to implement an existing algorithm for string comparison. The idea is to walk through both input and verse and compare them character by character, then come up with a similarity score. The hard part is to not end up with a very low score if there's a missing letter in one string. This is one of these things that is very easy for a human but really convoluted and messy for a computer.

[Perpetuall]

This is one of these things that is very easy for a human but really convoluted and messy for a computer.

Yep. That's why I was inputting everything one by one and trying to think of alternative ways I might spell or word something, and putting synonyms in as I went. Comparing translations for instance. Really tedious, but for the most part I found ways to make it work pretty well, as far as it being able to detect what I was trying to say as long as I had most of it there. It's just so time consuming, though. But I suppose with your module that will save me some time.

The parser can handle, like, 3-4 words at a time in each entry it seems. Longer than that and I start to get error messages.

I think using b is better than a because of the ability to swap out the phrases for alternative ones. It's more flexible, so if people remember, like, part of the NIV version of one verse and part KJV, it's okay, it can mix and match as long as you add synonyms.

Of course it would be a lot easier to just throw everything in and then add some type of code that says, "when you see the word "humble" add to your Humility score", but that would also be a really cheap way of doing it because you're bound to miss so many things that don't have any "keywords" in it. Like the parable of the sinner and the Pharisee in the temple. The entire point of the story is to be humble, but it never says that directly because it's trying to paint a narrative.

At one point I already compiled all the verses that would be relevant to each category when I was making the card game version of my game, and tried to use Biblehub to find a list of all the relevant verses, but to this day I still stumble upon verses that I missed that are totally relevant. They just don't show up because there's no direct references to certain words.

If I really want to be thorough about this, I think I'll have to keep taking it one verse at a time and tweak it as I go. But if I want to be more efficient, I think I'll have to put some thought into it and play around some more. Maybe I can find a balance.

As for the resolution problem, my game is set to run at 640x400. Which worked fine at first, and I could run it in fullscreen mode. It seems like whenever this happens, it's usually when I try to run the game from a backup file.

1366x768 is my desktop resolution

BTW Thanks very much for the program Khris. It makes this task a lot easier. Now, if only I could find a way to add a comma every three words, this would be a piece of cake.

[Khris]

I'm still not sure how this is supposed to work exactly. Spending all this time entering thousands of words is something I'd only do if I'm absolutely sure it's part of a working solution.

Also, you say you're getting errors if you're parsing longer sentences. Can you give examples of what exactly is in your word list and relevant code?

As for the resolution thing, which version of AGS are you using, and what settings do you have in your configuration?
I tried to run a 640x400 game in a 1366x768 virtual machine and all of the fullscreen modes worked more or less. (Although this might be different on actual hardware).

As for adding commas to your word list, again, I'm not sure what exactly you're doing. Or why you're adding three words at a time. I can try to help you but I need more information.

[Snarky]

For a task like this, I would absolutely try to read the Bible text from an external text file rather than copy it all into the AGS parser, which sounds like a massive time sink.

I think it would also be smart to try to separate the data from the logic, so that each can be iterated independently. If you find a better way to compare player input to the database of verses, you don't want to have to redo the whole database.

Finally, I would strongly recommend doing a small demo for people to playtest, of just this particular bit (with only a few verses) before you spend a whole lot of time copying the Bible into AGS: you might get feedback that makes you rethink/tweak the gameplay mechanic to something that needs to be implemented completely differently.

[Perpetuall]

Also, you say you're getting errors if you're parsing longer sentences. Can you give examples of what exactly is in your word list and relevant code?

I think it's due to the text not being inputted exactly as it appears in the parser. If I put the entire sentence into the list of words, but only type part of the sentence into the game parser, it crashes the game with an error message that says, "Such and such is not in the dictionary or is an ignored word". So for some reason that happens if a whole sentence is in the list of words, but only part of it is typed into the parser in-game.

That doesn't happen if I input everything into the word list three words at a time instead of putting entire sentences in. If I type in part of a sentence then, it just doesn't let me enter it, rather than crash the game.

This is how I'm structuring the text file that I put into the parser:

Code: ags

Psalm 1, Psalms 1

1, Blessed is the, man that walketh, not in the, counsel of the, ungodly nor standeth, in the way, of sinners nor, sitteth in, the seat, of the scornful 

2, But his delight, is in the, law of the, LORD and in, his law doth, he meditate day, and night 

3, And he shall, be like a, tree planted by, the rivers of, water that bringeth, forth his fruit, in his season, his leaf also, shall not wither, and whatsoever he, doeth shall prosper

4, The ungodly are, not so but, are like the, chaff which the, wind driveth away, wind drives away 

5, Therefore the ungodly, shall not stand, in the judgment, nor sinners in, the congregation of, the righteous 
 
6, For the LORD, knoweth the way, of the righteous, but the way, of the ungodly, shall perish, will perish

So each verse is numbered and considered one word with several synonyms.

In the global script:

Code: ags

else if (Parser.Said("Psalm 1:1")) 
	{ dPsalm1v1.Start();  virtue[29]++; vice[29]--; virtue[13]++; vice[13]--; virtue[20]++; vice[20]--; virtue[8]++; vice[8]--; virtue[5]++; vice[5]--; virtue[10]++; vice[10]--; GiveScore(1); txtParser.SetText(""); }
		else if (Parser.Said("Blessed is the man that walketh not rol")) 
	{ dPsalm1v1.Start(); virtue[29]++; vice[29]--; virtue[13]++; vice[13]--; virtue[20]++; vice[20]--; virtue[8]++; vice[8]--; virtue[5]++; vice[5]--; virtue[10]++; vice[10]--; GiveScore(1); txtParser.SetText(""); }
  else if (Parser.Said("Psalm 1:2")) 
	{ dPsalm1v2.Start(); virtue[22]++; vice[22]--; virtue[18]++; vice[18]--; virtue[13]++; vice[13]--; virtue[12]++; vice[12]--; virtue[8]++; vice[8]--; virtue[14]++; vice[14]--; virtue[26]++; vice[26]--; virtue[2]++; vice[2]--; GiveScore(1); txtParser.SetText(""); } 
		else if (Parser.Said("But his delight is in the law of the Lord and in his law doth he meditate rol")) //write to this length
	{ dPsalm1v2.Start(); virtue[22]++; vice[22]--; virtue[18]++; vice[18]--; virtue[13]++; vice[13]--; virtue[12]++; vice[12]--; virtue[8]++; vice[8]--; virtue[14]++; vice[14]--; virtue[26]++; vice[26]--; virtue[2]++; vice[2]--; GiveScore(1); txtParser.SetText(""); }
	else if (Parser.Said("Psalm 1:3")) 
	{ dPsalm1v3.Start(); virtue[17]++; vice[17]--; virtue[5]++; vice[5]--; virtue[2]++; vice[2]--; virtue[28]++; vice[28]--; virtue[29]++; vice[29]--; virtue[14]++; vice[14]--; GiveScore(1); txtParser.SetText(""); } 
		else if (Parser.Said("And he shall be like a tree planted by the rivers of water rol")) 
	{ dPsalm1v3.Start(); virtue[17]++; vice[17]--; virtue[5]++; vice[5]--; virtue[2]++; vice[2]--; virtue[28]++; vice[28]--; virtue[29]++; vice[29]--; virtue[14]++; vice[14]--; GiveScore(1); txtParser.SetText(""); }
	else if (Parser.Said("Psalm 1:4"))
	{ dPsalm1v4.Start(); virtue[13]++; vice[13]--; virtue[5]++; vice[5]--; GiveScore(1); txtParser.SetText(""); }
		else if (Parser.Said("the ungodly are not so but are like the chaff which the wind driveth away")) 
	{ dPsalm1v4.Start(); virtue[13]++; vice[13]--; virtue[5]++; vice[5]--; GiveScore(1); txtParser.SetText(""); }
		else if (Parser.Said("Psalm 1:5")) 
	{ dPsalm1v5.Start(); virtue[5]++; vice[5]--; GiveScore(1); txtParser.SetText(""); }
		else if (Parser.Said("Therefore the ungodly shall not stand in the judgement nor sinners rol")) 
	{ dPsalm1v5.Start(); virtue[5]++; vice[5]--; GiveScore(1); txtParser.SetText(""); }
		//example

Etc.

As for the resolution thing, which version of AGS are you using, and what settings do you have in your configuration?
I tried to run a 640x400 game in a 1366x768 virtual machine and all of the fullscreen modes worked more or less. (Although this might be different on actual hardware).

I forget whether this happened on AGS 3 or AGS 2.7 the first time, I keep switching back and forth between them since I am simultaneously working on two different parts of my game. Now it's happening on AGS 2.7. But I guess it's not a big deal, it just seems to happen when I try running a backup file. I'm going to be making a new game file and combining the two parts I'm working on anyway, so I think that will fix it.

Finally, I would strongly recommend doing a small demo for people to playtest, of just this particular bit (with only a few verses) before you spend a whole lot of time copying the Bible into AGS: you might get feedback that makes you rethink/tweak the gameplay mechanic to something that needs to be implemented completely differently.

Yes, that's why I thought I'd try to make a smaller chapter for MAGS using just the Psalter. I'm not sure I'm going to meet the deadline at this rate, though, and certainly not going to win. 

Unfortunately it's not just this part that's a massive time sink, the entire thing is a very tedious process. I'm also adding audio to each verse, and I'm having to categorize every verse into 30 Virtues, number them accordingly, and then script each one, plus make dialog files for them all. I calculated yesterday and just implementing the verses alone is taking me like 3-4 hours a day to keep up with my goal of 10 chapters per day (in 15 days at that rate I'll be done with Psalms). Which is unfortunate because that leaves me with a lot less time to focus on the artwork and narrative, which will probably end up being subpar. And that's a shame because improving my art skills is how I justified spending all this time on this project, lol. It's just a good thing I have a lot of free time right now.

I was also thinking, on a sidenote, how cool it would be to make something like this, but with poetry. (Haha I'm already coming up with new ideas when I'm still swamped with what I've got.) And then I wondered how you'd implement that, and how you could make a way for the parser to detect if what you're saying rhymes with something else you've said. That might be too complicated to do in AGS though. That's where AI would really come in handy.

EDIT: Hmm. Now my game is crashing whenever I have a typo, no matter how long the word entries are. Now it's just saying "It appears the game engine has not shut down properly." =/ I wonder what's causing that.

EDIT2: Fixed it. AGS just seems to have a problem handling too many words when the entries are so long, so shortening the longer sentences and adding "rol" keeps it from crashing.

EDIT3: In case anybody in the future out there is googling this and wondering for their own game, because I know I have certainly benefitted from obscure questions asked on these forums,  I discovered part of the problem was also that even when you delete the "ignored" words the game will crash if you try to enter them in, UNLESS you add these ignored words into your text parser as normal words manually. So if that is what you have done, you need to create a new game and copy that list of ignored words that are in there by default over to your game and add them to the text parser as "normal".

[Perpetuall]

Sigh.
I finally figured out a way to format the text file quickly and easily, but I can't even try it out because the file is too large and it crashes your program when I try to import it...

[Crimson Wizard]

Sigh.
I finally figured out a way to format the text file quickly and easily, but I can't even try it out because the file is too large and it crashes your program when I try to import it...

How large, and how exactly do you try to import it?

[Perpetuall]

It was the entire KJV in a text file, and I just used the import word list button on the website. It keeps giving me a "page not working" message.
I think I may have figured out a better way to do it, but I'm having trouble because the program isn't recognizing line breaks as a new word, for some reason...
@Khris is there another way to differentiate between words?

[Crimson Wizard]

So by "program" you mean the website, and not AGS?

Are you trying to post a big text on forums? Perhaps it would be easier to upload it to some file hosting and post a link here.

[Perpetuall]

I mean I'm trying to import it into Khris's program for the text parser... sorry for the lack of clarification.

[Crimson Wizard]

Oh you probably mean the https://khris.ddns.net/agsparser/ from one of the earlier posts?

[Perpetuall]

Yep. It's really strange, it's not recognizing new lines as the beginning of a new word anymore, it's just inputting the entire text file as one big word. Not what I wanted. 
I think the program I used to format the text document is using line breaks instead of hard returns. This is so frustrating, I'm so close to figuring it out...
EDIT: Yes! I figured out how to get it to input them as different words. Now, let's see if that works...
running into some technical difficulties
Ok, there's still some limitation on how many words the parser can handle. After 12 or so, it does nothing. I'll have to tinker with it more

EDIT2: Ok, so if anybody wants to know how to add an entire book to the text parser for any reason, this is what you have to do.
1. Go to https://onlinetexttools.com/remove-duplicate-text-words and input the entire text document into the input. Push "Enter" in the Output Word Delimiter box. It takes a second so don't give up. Once it loads, save the file as a text document. This eliminates all the extra words in the book so you can fit the entries into the parser, and separates them into different words-once I did this, the entire Bible was composed of only 12,000 or so unique words, so easily fits into the 30,000 limit.
2. Use Khris's program here to add it to your game and overwrite your game file. https://khris.ddns.net/agsparser/
3. Now you should be able to use all the words from the book in whatever combination you like in your game, and add any synonyms for alternate spellings and such. Oh, but, so far as I can tell, you need to add "rol" at the end of each line, like this:

Code: ags

else if (Parser.Said("But his delight is in the law of the LORD and in his law doth he meditate day rol")) 

I'm not sure why. It just won't accept it unless you do. And, also, it appears that there is still a limit to how many words you can enter in-about 16 appears to be the max, but cutting the sentence short and adding "rol" at the end will make it so it still recognizes it, and as long as nobody knows that it's autocompleting the sentence for them after that, what does it matter? There doesn't seem to be, as far as I can tell, any way around this limit.

Anyway, THANKS @Khris! That saved me a lot of trouble.

Errr, one more thing: Now I have to do the scripting. I vaguely remember there being some feature in Excel to input a list into paragraphs, and since I haven't used Excel since highschool, I have no idea if I'm remembering that right or how to do it. You seem like somebody who codes a lot. Is there a feature out there to make it simpler to write lines of script that are almost all the same but just slightly different? If I could just find a way to make 31,000 copies of the code above and just change the text inside the quotations, that would be amazing. Surely there's a way?

[Khris]

Excel is the wrong tool; just use any language that can read and output text.
However you should make sure your approach works first. Afaik you cannot have thousands of else ifs in a row. As for automating, I'd also automate the commands that are run if a match is found.

Scratch that; I'd separate the two things completely.
First, write code that finds which book, chapter and verse was typed. (I'd use a separate text file for each book, btw)

Once you have "Psalms", 1 and 5 stored in three variables, proceed from there. Anything else is madness, especially putting 31,000 else if lines directly in a script. That is the worst possible approach no matter how you slice it.

[Perpetuall]

Yes, else if's don't work, but so far if's do. Not that that's the best solution either.
I'm not sure what you mean about using variables for this, can you give me an example?

[Khris]

Very generally put, using pseudo-code:

Code: pseudo

String book;
int chapter, verse;

loop over book text files
 loop over chapters
  loop over verses
    if (match found) set book, chapter, verse and break out of loops

set vice / virtue variables based on book, chapter, verse
clear input
start next dialog

Imagine the nightmare of finding out you also need to do something else and having to amend 31000+ lines of code. (In addition to the unimaginable horror of first writing these 31000+ lines of code.)
You need to come up with some system that is much more amenable to later changes.

[Perpetuall]

I agree, but I admit I'm having trouble grasping the precise way to implement this pseudo-code. You have to remember I have very little experience with scripting, what I do know is basically from just dabbling with things and seeing how they work and reading what other people did here and there. So a lot of the time I read your suggestions and think, "That's genius, but I have no idea where to begin doing that myself". Then in the meantime I'll have cobbled together some other wacky way of doing it that's totally different from the idea I first began asking about.
I really appreciate your input but I'm afraid I'm going to need a walkthrough, especially when it comes to using strings and stuff, and where I'd begin using that method.