Recommendation: GSI CLEP (v1)
Channel and message protocol for efficient and flexible LSL chat communication
The Global Scripting Institute (GSI) is an informal organization of Second Life® users that design and test standards for efficient, flexible, and readable scripts in Second Life. "Second Life®" and "Second Life Grid™" are trademarks of Linden Research, Inc., d/b/a Linden Lab. The Global Scripting Institute and its catalog are not affiliated with with or sponsored by Linden Research.
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in RFC 2119.
The Chat link_message Extension Protocol (CLEP) is a protocol for inter-script chat communication in Linden Scripting Language (LSL).
CLEP transmits serialized link_message Extension Protocol (LEP) messages (or any other raw strings) over chat to scripts in other linksets in the same region. LEP is a data format that allows scripts to rapidly route messages to individual scripts in a linkset. Combined, LEP and CLEP allow scripts to transmit LEP messages to any other script in the region that is listening for CLEP messages on a shared service and domain, which are replacements for channel integers.
CLEP uses llRegionSay (region-wide broadcasts), llRegionSayTo (targeted to a specific prim in the same region), and llShout (targeted to a specific prim presumed to be in an adjacent region).
CLEP message recipients MUST open a llListen listener to receive CLEP messages, as follows:
llListen(channel, "", "", ""); // listen to all names, UUIDs, and messages on channel
| Placeholder | Type | Description |
|---|---|---|
channel |
integer | MUST be the result of the CLEP Channel Hash Function described below. |
For each CLEP service and domain combination the script wants to listen to (see below), it must call llListen for the channel used by that combination. The script MAY store the return value of llListen to use later with llListenRemove.
CLEP messages MUST be sent either to a specific prim or all prims in a region. (CLEP implementations filter out messages not targeted to them in two ways: channel separation, and parameter discrimination, both of which are described later.)
Messages sent to a specific prim listening to a channel MUST be constructed as follows:
llRegionSayTo(target_prim, channel, serialized_message);
Notwithstanding the above, transmitting scripts MAY first confirm that target_prim exists in the region, then, if target_prim does not exist in the region, MAY instead construct the message as follows to allow for cross-region-border communications, or MAY drop the outgoing message entirely:
llShout(channel, serialized_message);
Messages sent to all prims listening to a channel in a region MUST be constructed as follows:
llRegionSay(channel, serialized_message);
The message will only be delivered if the recipient has called llListen with the same channel.
| Placeholder | Type | Description |
|---|---|---|
target_prim |
key | MUST be the UUID of the target prim, if sending a message to a specific prim. If broadcasting to all prims, this placeholder is not used. |
channel |
integer | MUST be the result of the CLEP Channel Hash Function described below. |
serialized_message |
string | MUST be the serialized message, generated by the CLEP List Serialization Function, both described below. |
The CLEP Channel Hash Function (CHF) generates an integer channel based on a string service and string domain. This facilitates CLEP's channel separation, which reduces the number of events triggered when a large collection of objects is broadcasting messages to each other on a common hard-coded channel integer.
The Channel Hash Function MUST be defined as follows:
integer channel = llHash(service + domain) | 0x80000000;
This accomplishes the following:
service and domain strings, which is returned as a 32-bit integer.| Placeholder | Type | Description |
|---|---|---|
service |
string | MUST be a string of any value. The value MUST NOT contain any newline ("\n") characters. This value MUST be shared among all scripts that use any single implementation of CLEP. This is to prevent crosstalk between different systems listening on the same domain. For example, if two script systems are in a single prim and both use the prim's UUID as a domain, this value lets both mutually exclude each others' CLEP traffic. |
domain |
string | MUST be a string of any value. The value MUST NOT contain any newline ("\n") characters. This value can be treated as an arbitrary "string channel". By separating CLEP traffic in each domain to a different integer channel, CLEP messages from other domains are filtered server-side - messages not sent to the same domain will not trigger a listen event in the first place.A pre-shared string, the prim's UUID, another prim's UUID, or some combination of the three, is RECOMMENDED. |
Note that the Channel Hash Function is not designed to be secure or private. The SDBM algorithm is not cryptographically safe and collisions are extremely rare, but possible given the resulting 31-bit channel space.
Also note that service and domain perform the same task of message discrimination - not only does the recipient filter received messages where either service or domain does not match an expected value, but both values are used to generate the channel hash, so the recipient will not receive the message at all unless both service and domain match with respect to a channel that the script is listening to. This distinction is only to aid the script in delineating between a broader "service" (such as the name of a product line, which should never change) and a specific "domain" (a further limiting identifier for a group of objects that need to be networked together specifically, which could be changed by the end-user, depending on implementation specifics).
The CLEP List Serialization Function (LSF) serializes a list with any characters into a string with the ability to detect truncation in transit. CLEP uses the LSF to serialize LEP's target_script, flags, parameters, and data into a CLEP message, then uses the LSF to serialize CLEP's service, domain, target_prim, type, and message into a string sent via chat.
The LSF MUST take an input list with any or no elements, then for each element, pass it through llEscapeURL and append it and "," to the end of an output string.
The LSF takes this approach for several reasons:
The serialized_message sent for a CLEP message is a list, serialized into a string using the LSF. The elements are:
["CLEP", service, domain, target_prim, type, message]
| Placeholder | Type | Description |
|---|---|---|
service |
string | MUST match the service used by the recipient when listening to a CHF channel and by the sender when sending a message. |
domain |
string | MUST match the domain used by the recipient when listening to a CHF channel and by the sender when sending a message. |
target_prim |
key | MUST be one of the following:
|
type |
string | MUST be one of the following:
|
message |
string | Depends on the value of type. For "LEP" message types, MUST contain a LSF-serialized LEP message (see below). For "" (raw) message types or any other message types, MAY contain any value. |
LEP messages MUST be serialized into a CLEP message using the LSF on the following list:
[flags, llDumpList2String([llGetScriptName(), target_script] + parameters, "\n"), data]
| Placeholder | Type | Description |
|---|---|---|
flags |
integer | MUST be the LEP message's flags. |
target_script |
string | MUST be the LEP message's target_script. |
parameters |
list | MUST be the LEP message's parameters. |
data |
string | MUST be the LEP message's data. |
Note that CLEP does not support the target_link LEP parameter, because link number discrimination is not particularly useful considering the recipient must manually listen for CLEP messages anyway.
For more information on these values, refer to the link_message Extension Protocol recommendation.
CLEP messages received are not necessarily targeted to the script that receives them. This can be because of channel conflicts caused by hash collisions (in the extremely rare case where two non-matching service and domain names nonetheless return the same hash) or due to how the CLEP message must be sent (particularly by shouting for cross-region communication).
The CLEP message recipient SHOULD filter out any messages that have a non-matching service, domain, target_prim, or target_script. This parameter discrimination, alongside channel separation, is how CLEP messages are processed with minimal script cycles. An example implementation of this is below.
A full reference implementation of a CLEP message sender is below:
// call this function to send a LEP-over-CLEP message
clep_send_lep(
string service,
string domain,
string target_prim,
string target_script,
integer flags,
list parameters,
string data
)
{
_clep_send(
service,
domain,
target_prim,
"LEP",
_clep_serialize([
flags,
llDumpList2String([llGetScriptName(), target_script] + parameters, "\n"),
data
])
);
}
// call this function to send a raw CLEP message
clep_send_raw(
string service,
string domain,
string target_prim,
string data
)
{
_clep_send(
service,
domain,
target_prim,
"",
data
);
}
// internal functions
// encapsulate parameters into a valid CLEP message
_clep_send(
string service,
string domain,
string target_prim,
string type,
string data
)
{
_clep_out(
target_prim,
llHash(service + domain) | 0x80000000,
_clep_serialize([
"CLEP",
service,
domain,
target_prim,
type,
message
])
);
}
// route the CLEP message depending on target_prim
_clep_out(
string target_prim,
integer channel,
string message)
{
if (target_prim == "") llRegionSay(channel, message);
else
{
list details = llGetObjectDetails(target_prim, [OBJECT_OWNER]);
if (details == []) llShout(channel, message); // target_prim is not in region, so try shouting
else if (llList2String(details, 0) == llToLower(target_prim)) llOwnerSay(target_prim + " could not be targeted for CLEP because it is an avatar");
else llRegionSayTo(target_prim, channel, message); // target_prim is a prim in this region
}
}
// List Serialization Function
string _clep_serialize(
list in
)
{
string out;
integer i;
integer l = llGetListLength(in);
for ( i = 0; i < l; i++ )
out += llEscapeURL(llList2String(in, i)) + ",";
return out;
}
default
{
state_entry()
{
clep_send_lep(
"Service Name", // service
"Domain Name", // domain
"", // target_prim
"", // target_script
0, // flags
["parameters"], // parameters
"data" // data
);
}
}
A full reference implementation of a CLEP recipient is below:
string CLEP_SERVICE = "Service Name";
string CLEP_DOMAIN = "Domain Name";
// List [Un]Serialization Function
string _clep_unserialize(
string in
)
{
if (in == "") return []; // empty list
list out = llCSV2List(in);
if (llList2String(out, -1) != "") return []; // input string was truncated
// WARNING: if the string is truncated exactly after a comma, this will not be caught!
list unescaped;
integer i;
integer l = llGetListLength(out) - 1; // don't bother with final padding element
for (i = 0; i < l; i++)
unescaped += [llUnescapeURL(llList2String(out, i))];
return unescaped;
}
default
{
state_entry()
{
llListen(llHash(CLEP_SERVICE + CLEP_DOMAIN) | 0x80000000, "", "", "");
}
listen(integer channel, string name, key id, string message)
{
list parts = _clep_unserialize(message);
if (llList2String(parts, 0) != "CLEP") return; // not a valid CLEP message - can also call other code here, such as for llDialog/llTextBox responses over a CLEP channel to avoid multiple listens
// split out CLEP parts into parameters
string service = llList2String(parts, 1);
string domain = llList2String(parts, 2);
string target_prim = llList2String(parts, 3);
string type = llList2String(parts, 4);
message = llList2String(parts, 5);
// you can handle the CLEP message any way you want at this point, but the below is RECOMMENDED
if (service != CLEP_SERVICE || domain != CLEP_DOMAIN) return; // channel interference
if (target_prim != "" && target_prim != (string)llGetKey()) return; // llShout interference from a nearby sender trying to reach a different recipient not in the same region
if (type == "LEP")
{ // LEP message processing
parts = _clep_unserialize(message);
integer flags = (integer)llList2String(parts, 0);
list parameters = llParseStringKeepNulls(llList2String(parts, 1), ["\n"], []);
string source_script = llList2String(parameters, 0);
string target_script = llList2String(parameters, 1);
parameters = llDeleteSubList(parameters, 0, 1);
string data = llList2String(parts, 2);
// you can now do anything you like with flags, source_script, target_script, parameters, and data
// you can also refer to service, domain, and target_prim if it's useful (these are CLEP-specific)
llOwnerSay("Received LEP message via CLEP with data: " + data);
}
}
}
Why are channels hashed? Isn't it better to have a pre-shared channel?
Per the Second Life Wiki, the Second Life server processes every chat message by first checking that it matches the channel of any open listens in the region, then performs some additional checks (self-chat, distance, other llListen filters). If - and only if - those checks pass, a listen event is added to the script's event queue.
CLEP enforces channel separation using hashing to take advantage of the performance gains offered by this check. Traditionally, LSL scripts use a pre-shared channel integer for a certain product. If a shared channel is used, any broadcasts that need to be done via llRegionSay, llSay, llShout, or llWhisper are sent to all scripts listening to that channel.
Advanced networks of scripts tend to use multiple channels to offset this. However, sharing multiple integers as channel numbers is complex, and the integers don't themselves have any substantial intrinsic meaning.
CLEP does something similar, except that the channel numbers are deterministically pseudo-random by use of a hash function. That way, instead of randomly coming up with a channel number, CLEP channels are effectively defined as strings and the actual integer channel used internally is deterministic but irrelevant.
As a result, CLEP encourages the use of multiple channels (via the domain value) for complex scripts, which subtly enforces channel separation and reduces script impact overall by reducing unnecessary listen events.
What's the difference between raw and LEP messages?
Raw CLEP messages are treated like regular chat/listen message strings with some extra encapsulation for routing purposes.
LEP messages allow you to send a string (data), a list (parameters), and several bitwise flags to facilitate hierarchical request-response protocols.
There's generally no reason to use raw CLEP messages instead of LEP-over-CLEP unless you are extremely memory-limited and only need to send simple commands.
How do I discriminate between CLEP messages and plain chat?
All CLEP messages have a "CLEP" header. The above reference implementation returns the event immediately if this header is not detected, but you may want to instead parse the message as a plain string. This is particularly important if your script uses llDialog/llTextBox*, or listens for chat.
* Consider using something like (llHash((string)llGetKey()) | 0x80000000) to generate a likely-unique channel number for any specific prim.
CLEP was authored by Nelson Jenkins on behalf of GSI.