Sockets for AGS (alpha 1) -- testers required!

[Wyz]

Yes! Sockets for AGS. There are already a variety of socket plugins out there (one or two by me ) for AGS but this one tries to be as general as possible. However it is not yet finished! The reason why: it needs more testing; I have tested it but to test this plugin properly I actually need other people to play around with it.

So that is where the community comes in! I need a couple of developers that already have some understanding of sockets to test it. You can leave all your findings, comments, questions and requests here in this topic. I'll post updates when I have them also here. If you have something to showcase you're also free to post in this topic. Since I don't have documentation (except for the auto-completion hints which you should try) if you're unsure how a functions works or what it returns you can also post it here; it will also give me an indication of where to put the focus when writing the manual.

So I proudly present:

agsSock Alpha 1

Warning: This plugin is not compatible with the old plugins (including mine). If you're already using a sockets plugin in a project make a backup of the old plugin!

So the current state: Some features are not yet implemented or optimized and proper documentation (a manual) is lacking. Other then that it is presumed fully functional.

Some of the characteristics of this plugin (design decisions):

General
The plugin tries to be close to Berkeley sockets (for those who know it) to take advantage of a widely used standard. Yet it will also wrap around certain things to make it more convenient for use in AGS. The biggest change is that it is object orientated.

Object orientated
AGS has been object orientated for the past few versions so let's take advantage of that. Sockets and addresses are objects which also keep internal records. This makes it possible cache information for efficiency reasons. It keeps track of validity and errors for each socket independently. It also has a built-in buffer.

Buffered
Every socket has a buffer for incoming messages meaning that you don't have to worry about read errors except those that are fatal. The buffers and error states are updated in a separate thread.

Threaded
The plugin is threaded: that means that even if AGS is waiting for user input the plugin will still process sockets in the background like reading incoming data or processing errors. It can do this with a single thread that is only active when something actually happens. This makes receiving messages effectively non-blocking.

Non-blocking
The entire plugin tries to be non-blocking. Blocking is where the application will stop and wait till it can complete the action (like waiting till a message was sent so it can receive it or a connection was made). Blocking is not very convenient for use in AGS because that will make the game unresponsive. Instead all methods will return unsuccessfully (but without an error number beware for amusing errors messages) if nothing happened.
There are two exceptions:

• Resolving addresses (converting addresses to string and the other way around)
• Connecting synchronously (default)

However it is possible to work around both in cases where it becomes a problem.

A standard
The plugin is based on a standard but also tries to be one itself. When it's stable my intentions are to also make modules for commonly used communication protocols like HTTP. I'll try to make a proper manual and provide examples for those who want to use it themselves. For now all I have is auto completion hints though.
It also tries to be as portable as possible but for now I will only release windows binaries (it should be posix compatible but I've never tested it).


I hope for now people play around with it and it will find an use in the future. It can also be a stepping stone for those who want to learn more about computer networks. Maybe we'll see the first multi-play adventure game. I'm excited to see all that!

[DoorKnobHandle]

Here is an open-source sample that might help you get started: http://dkh.agser.me/sock.zip (AGS 3.2.1)

To test this out, start Compiled/sock.exe twice, click on "Server" in one instance of the game, click on "Client" in the other. The connection should be set up and you should be able to chat with the other game!

This is not useful in itself but it shows the steps (look at TCP.asc) necessary to get started with this plugin!

EDIT: If anybody wants to update/improve/document this sample, go right ahead please!

[Wyz]

Thanks for the demo program dkh, look pretty good.

I guess I'll also a little example so people get a feeling how things work. This is a multi-client chat server you can use with a telnet client. Simply connect with your client to localhost.

Here it is:

Code: ags

// room script file

Socket *server;

//- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 

struct Client
{
  Socket *sock;
  String buffer; // unprocessed input
};

#define MAX_CLIENTS 10
Client client[MAX_CLIENTS];
int clients = 0;

//------------------------------------------------------------------------------

void AddClient(Socket *sock)
{
  client[clients].sock = sock;
  client[clients].buffer = "";
  clients++;
}

//------------------------------------------------------------------------------

void RemoveClient(int id)
{
  clients--;
  client[id].sock   = client[clients].sock;
  client[id].buffer = client[clients].buffer;
  client[clients].sock   = null;
  client[clients].buffer = null;
}

//------------------------------------------------------------------------------
// Adds input to the client's process buffer

void BufferPush(this Client *, String msg)
{
  this.buffer = this.buffer.Append(msg);
}

//------------------------------------------------------------------------------
// Fetches the next command from the process buffer (if there is one),
// otherwise: returns null.
// Note: commands are terminated by a carriage-return and linefeed character.

String BufferPop(this Client *)
{
  int i = this.buffer.IndexOf("\r\n");
  if (i < 0) return null;
  
  String str = this.buffer.Truncate(i);
  this.buffer = this.buffer.Substring(i + 2, this.buffer.Length - i - 2);
  return str;
}

//------------------------------------------------------------------------------

void SendAll(String msg)
{
  int i = clients;
  while (i)
  {
    i--;
    client[i].sock.Send(msg);
  }
}

//------------------------------------------------------------------------------

function entry() // Enters room after fade-in
{
  SetMultitaskingMode(1);
  server = Socket.CreateTCP();
  
  if (!server.Bind(SockAddr.CreateFromString("telnet://0.0.0.0")))
  {
    Display("Unable to bind to port: %s", server.ErrorString());
    return;
  }
  
  if (!server.Listen())
  {
    Display("Unable to listen: %s", server.ErrorString());
    return;
  }
}

//------------------------------------------------------------------------------

function loop() // Repeatedly execute
{
  Socket *sock;
  
  // Debug text
  if (server == null || !server.Valid)
  {
    lblOutput.Text = String.Format("Disconnected!", server.Local.Address, clients);
    return;
  }
  lblOutput.Text = String.Format("Listening: %s, %d clients", server.Local.Address, clients);
  
  // Accept new connections
  sock = server.Accept();
  if (sock)
  {
    // Client connected
    AddClient(sock);
    sock.Send("Please enter a nickname: ");
  }
  else if (server.LastError)
    Display("Unable to accept connection: %s", server.ErrorString());
  
  // Process messages
  String msg;
  int i = clients;
  while (i)
  {
    i--;
    
    msg = client[i].sock.Recv();
    if (msg != null && msg != "") // Non-empty message arrived
    {
      client[i].BufferPush(msg);
      
      msg = client[i].BufferPop();
      while (msg != null)
      {
        // Incomming message
        if (client[i].sock.Tag == "")
        {
          client[i].sock.Tag = msg;
          SendAll(String.Format("%s entered.\r\n", client[i].sock.Tag));
        }
        else
          SendAll(String.Format("[%s] %s\r\n", client[i].sock.Tag, msg));
        msg = client[i].BufferPop();
      }
    }
    else if (msg != null && msg == "") // Empty message arrived
    {
      // Client closed connection
      SendAll(String.Format("%s left.\r\n", client[i].sock.Tag));
    }
    else if (client[i].sock.LastError) // Trouble arrived
    {
      // Client had an error
      SendAll(String.Format("%s was disconnected: %d\r\n", client[i].sock.ErrorString()));
    }
    
    // Clean up lost clients
    if (!client[i].sock.Valid)
      RemoveClient(i);
  }
}

[dbuske]

I never heard of a socket. Is it a network protocal?

[Construed]

Beautiful work!
I'm a little too far with the old plugin to switch over at the moment, but I will dabble with it soon and see if I can help debug.

[Wyz]

Thanks! In a few weeks when I'm not as busy as right now I will make a few more and more practical examples; I guess that will generally be more useful then the very bare state the plugin is in right now.

I never heard of a socket. Is it a network protocal?

Sockets are the end points of a network protocol; if you view a network protocol as a cable then the sockets are the places where the cable is connected. That way you can tell what the network is connected to and send and receive messages to and from the other side.

[RickJ]

Wyz,

Bless you sir!  I shall take this us next week.

Thanks

[Crimson Wizard]

Works with 3.3.0 beta (dkh's demo script).

I think I will try making a simple multiplayer game with this.

[Crimson Wizard]

Every now and then I am getting an error when starting AGS:

There was an error loading plugin 'agssock.dll'.

Unable to load plugin 'agssock.dll'. It may depend on another DLL that is missing.

After some time (or maybe some actions) it loads fine.


On other topic, is there a way to a) refuse client connection and b) drop previously connected client?

[Construed]

I don't exactly know how this version works but some commands from the irc version:

Code: ags

void Connect(String nick)
{
  if (!IRCConnect(SERVER))
  {
    Display("You failed to connect, Please try again.");
    return;
  }
  
  if (String.IsNullOrEmpty(nick))
    nick = DEFAULT_NICK;

  nickname = nick;

  channel = Room.GetTextProperty("channel");
     
  IRCNick(nick);
  IRCUser(nick, "agsIRC");
  player.Name = nick;
  
  connected = true;
}

Then use:Connect(); or Connect(nick);

And for disconnect:


void Disconnect()
{
  connected = false;
  initiated = false;
  IRCDisconnect();
}
and call the:
Disconnect();

I hope you figure it out my friend, I also want to test this version soon!

[Crimson Wizard]

Hmm, no, this plugin has completely different set of commands, nothing with "IRC".

E: I see how I can shutdown the socket ran as "host", and I see how I can disconnect client being the client, but I do not see how to drop client connection when being the host (i.e. "kick" another player from the game). Or maybe I am missing something.

[Construed]

Ah, yes... Much to advanced for me at the moment.
You should PM WYZ and tell him about this post, Sometimes its hard to notice a topic amongst all the other.
Hope you figure it out bud, I plan to dabble with this too when I get some spare time

[Wyz]

Sorry for my very very late response!

Every now and then I am getting an error when starting AGS:
(...)

I've heard about this error before but have been unable to reproduce it myself. One person claimed it had something to do with steam running in the background but I have no way of confirming this unfortunately. If this happens regularly place let me know.

On other topic, is there a way to a) refuse client connection and b) drop previously connected client?

You can refuse incoming connections but only without knowing what remote host is trying to connect. This mechanism is mainly used to stop accepting any new connections when the server has met its capacity. The way this works is that there is a queue on the listening end that holds new connections. When this queue is full new connections will receive a 'connection refused'. The size of this queue is called backlog and can be specified by the Listen method.
Connections will be left waiting in the queue until you call the Accept method. At that point you grant them a connection and you also get their ip.

So how to refuse certain connections? Well you accept then an then immediately close them optionally sending some error along. So that bring me to part (b) of your question: how to close them?
Accept returns a new Socket object representing this new connection. Call their Close method to close them gracefully (connection closed) or dereference them (set the pointer to null) so that AGS' garbage collector deletes them which closes them in not so a graceful way (connection reset).

I hope that answers your questions and again sorry for my late response.

[Khris]

First of all, amazing plugin!
I also got the "Unable to load plugin 'agssock.dll'." error about an hour ago, couldn't resolve it, restarted my PC and it was gone.

But, I'm more interested in the proper way to read HTTP data. I'm sending a GET request, then simply listen in a while loop until Recv() returns something other than null. The thing is if I run this on my old laptop, I'm not getting the complete reply in one go; right now I'm using two nested loops and a timeout to make sure I'm getting the entire message. I noticed that there's a 0 (zero) after the content; do I have to keep reading lines until I get to the 0? What if my data actually contains \r\n0\r\n? Unlikely but possible, right?
Basically, how do I read the entire content without essentially halting my game for a few seconds? Assuming this is possible, of course.
Or do I have to live with this and just wait for either the zero or a timeout?

[Wyz]

Thanks! Thanks for trying it!

The plugin uses an internal buffer (other then the buffer at system-level) to store incoming data until you use Recv. It processes incoming data as it comes in a separate thread and puts it in the buffer. When the connection was closed an empty string is returned by Recv. Since http servers typically close the connection by default after sending the response you can assume you received everything after the connection has been closed (thus you get a empty string from tRecv). You can force this behaviour by sending the header Connection: close along your request.

Because the plugin is threaded you don't have to do a wait loop and you can put the Recv calls in execute_repeatedly. When it returns null and the error code is 0 it means there has not yet been a response for the server so you can skip processing that frame.

I don't know exactly what the 0 after the content is. When you're using a http 1.1 request the server should return something like:
STATUS\r\nHeader1: ...\r\nHeader2: ...\r\n\r\n<data>
The request should look something like his: (note the extra \r\n at the end)

Code: ags

GET /index.html http/1.1
Host: www.someurl.com
Connection: close

So yeah everything after the first double CRLF should be data if all is well. Let me know if that works.

[DoorKnobHandle]

I also get the additional 0 as well (and an extra b for some reason?):

Code: ags

HTTP/1.1 200 OK
Date: Sat, 01 Feb 2014 16:39:10 GMT
Server: Apache
X-Powered-By: PHP/5.4.21
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html

b
Hello Hans!
0

This is me sending a similar 1.1 request as the one you posted above, Wyz (except it should be forward slash at the end of the first line I believe) to a php script on dkh.agser.me. I was also wondering where those extra characters come from, no idea so far. The php script obviously only does "print 'Hello Hans!'"

[Khris]

I'm getting a 59 before my highscore string.
First I thought it's the length, but the String's length is actually 89. I thought about your b and suddenly the lightbulb came on: it is the length, but in hex

Edit:
Changed some things around, implemented proper parsing.
I'm getting this:

Code: txt

HTTP/1.1 200 OK
Date: Wed, 19 Feb 2014 02:04:49 GMT
Server: Apache
X-Powered-By: PHP/5.4.21
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html

51
1. selmiak: 108[2. Khris: 77[3. Arnjeir: 39[4. Peder Johnsen: 25[5. klotzkopp: 20

Flawless!
Thanks again Wyz, great job!

[Wyz]

Oh I know what that is; the response header gave it away Yes that is chunking, a way for http servers to split up the message body in chunks. No way around it unfortunately and some servers are particularly eager to use it. Nice! Looks like you already found out but for the record you can read here how to parse it. Transfer encoding can be a pain sometimes.

(btw I fixed the slash, that was a typo )

[Khris]

I'm trying to get binary data. Apparently I can't do that using Strings though because those omit 00 chars.
Tried to do it with RecvData() instead and I'm getting nothing, it always returns null. Is this not implemented yet?

Edit: nevermind, just discovered a really stupid typo in my code
Edit: actually, still not working...

[Wyz]

Yes you're right, RecvData has not yet been implemented. It will become available in the next version but for now it doesn't work unfortunately.
Maybe Base64 encode the data for now? I have a base64 decode module you can use if you want.