Growing Yggdrasil, the World Tree, with Ash

2026-04-05T00:00:00+00:00

Declarative programming can be a powerful paradigm for organizing software systems. By defining the business processes once, we ensure there is a single source of domain knowledge. From this foundation, we can derive other parts of the system such as API endpoints, database schemas, and even user interfaces. This approach reduces repetition and helps prevent bugs caused by misaligned domain models.

The Ash Framework</a> seems like an excellent way to see declarative programming in action. Written in Elixir</a>, it allows you to describe your domain in a consistent and expressive way, from which it can automatically generate data layers, REST or GraphQL APIs, and admin interfaces. With Ash, you define what your application should do, and the framework takes care of how to make it happen. It can derive JSON REST endpoints, handle validation, manage persistence, and provide authorization logic, all from the same declarative definitions.

I am new to Ash and I tend to learn best by writing things out, so this article is as much for me as it is for you. Rather than trying to understand everything up front, I prefer to get hands-on quickly with a small, self-contained project. We’ll start with the basics here, and if things go well, expand on it in a follow-up or two.

Given the name Ash, it felt appropriate to build something inspired by the gigantic ash tree of Norse mythology: Yggdrasil, the World Tree</a>.

An image of a Yggdrasil the world tree as a cybernetic Ash tree.</h4> </figcaption> </figure>

Yggdrasil is said to connect the Nine Worlds of Norse cosmology</a>, though the exact number and nature of these worlds vary between sources. Each world has its own nature, inhabitants, and relationships with the others, making it an ideal metaphor for exploring how Ash models resources, attributes, and relationships. In this project, we will create a domain model for these concepts with Ash and derive a JSON REST API from them.

The first step is getting started with a basic Ash project for which we will use the Igniter tool. (I will assume Elixir is already installed, but if not, see the Elixir Install page</a> for instructions). This is used for project setup and code generation, which will help us get started a lot quicker.

To start off following command will install Igniter:

mix archive.install hex igniter_new</code>

Once we have Igniter, the next step is creating a new project in the yggdrasil</code> directory, adding Ash, and moving into it.

mix igniter.new yggdrasil --install ash && cd yggdrasil</code>

This will land us in a new Elixir project directory with Ash installed. From this seed we will evolve our application to represent the worlds and characters of Norse mythology.

The newly created project comes with a hello</code> function in the lib/yggdrasil.ex</code> module. Let's try it out in the iex</code>, the Elixir interactive shell which we can start with:

iex -S mix</code>

Running it will bring us into the shell, where we can run the hello</code> function in the yggdrasil</code> module:

iex(1)> Yggdrasil.hello()
</span>:world
</span></code></pre>
We now have a hello world, but in yggdrasil we want to represent the worlds</em> of Norse mythology that are linked by Yggdrasil. The first thing we need for this is a Domain</em>. This will function as a container for the various concepts, such as the worlds, that we will introduce later.</p>
For simplicity's sake, first we will replace the contents of lib/Yggdrasil.ex</code> with the following:</p>
defmodule </span>Yggdrasil </span>do
</span>  </span>@moduledoc """
</span>  The Yggdrasil domain — acts as the trunk of the tree
</span>  and organizes all resources like World and Character.
</span>  """
</span>
</span>  </span>use </span>Ash</span>.</span>Domain
</span>
</span>  resources </span>do
</span>    # Resources will be registered here
</span>    resource </span>Yggdrasil</span>.</span>World
</span>  </span>end
</span>end
</span>
</span></code></pre>
We create a file lib/resources/world.ex</code> with the following contents:</p>
defmodule </span>Yggdrasil</span>.</span>World </span>do
</span>  </span>@moduledoc """
</span>  A resource representing a world in Yggdrasil.
</span>  """
</span>
</span>  </span>use </span>Ash</span>.</span>Resource</span>,
</span>    # in-memory store
</span>    </span>data_layer: </span>Ash</span>.</span>DataLayer</span>.</span>Ets</span>,
</span>    </span>domain: </span>Yggdrasil
</span>
</span>  actions </span>do
</span>    create </span>:create </span>do
</span>      accept [</span>:name</span>, </span>:description</span>]
</span>    </span>end
</span>
</span>    update </span>:update </span>do
</span>      accept [</span>:description</span>]
</span>    </span>end
</span>
</span>    # Provide default actions
</span>    defaults [</span>:read</span>, </span>:destroy</span>]
</span>  </span>end
</span>
</span>  attributes </span>do
</span>    # Primary key
</span>    uuid_primary_key </span>:id
</span>
</span>    # World name and description
</span>    attribute </span>:name</span>, </span>:string</span>, </span>allow_nil?: false</span>, </span>public?: true
</span>    attribute </span>:description</span>, </span>:string</span>, </span>public?: true
</span>  </span>end
</span>end
</span></code></pre>
And finally </p>
config </span>:yggdrasil</span>, </span>:ash_domains</span>, [</span>Yggdrasil</span>]
</span></code></pre>
With these files in place, we now have a minimal Ash domain containing a single resource: World. Let’s take a moment to unpack what we just created before moving on.</p>
At the top level, Yggdrasil acts as our domain, the trunk of our system. It brings together all the resources that make up the application and defines how they relate to each other. Right now, our domain only includes one resource, Yggdrasil.World, but we’ll add more later.</p>
The Yggdrasil.World module itself is declared as a resource. In Ash, a resource is the fundamental building block. It describes a specific type of data and what can be done with it. Instead of writing separate schemas, changesets, and controllers, we declare everything about a resource in one place, and Ash takes care of the details.</p>
Our World resource uses the Ash.DataLayer.Ets data layer, which stores data in Elixir’s in-memory ETS tables. This setup is fast and simple, making it perfect for early experimentation, though data won’t persist between runs. Later, this can be swapped out for a different data layer to gain full persistence. The argument domain: Yggdrasil</code> connects the resource back to the domain we just defined so that the framework knows where it belongs.</p>
Inside the actions block, we declare what operations are available for this resource. The create action accepts a name and a description, while the defaults [:read, :destroy]</code> line automatically adds the standard read, update, and delete actions. There’s no need to write any manual CRUD logic—Ash generates it for us.</p>
The attributes block defines the structure of each world. Every world has a UUID primary key (:id</code>) and two fields, :name</code> and :description</code>. The :name</code> attribute is made required using (allow_nil?: false)</code>, ensuring that each world must have one. Both attributes are marked public?: true</code> so they appear in APIs and outputs.</p>
Finally, the configuration line we added tells Ash which domains to load when the application starts. Without this, the framework wouldn’t know about our new resource.</p>
At this point, our small Ash tree has already taken root. We’ve declared the first piece of our domain, and Ash now knows how to create, read, update, and delete worlds. Let’s see that in action next by exploring our resource interactively in iex.</p>
First, start the interactive shell from your project root:</p>
iex -S</span> mix
</span></code></pre>
Once inside iex, we want to create our first world Asgard, the shining realm of the gods, with the following command:</p>
asgard = (
</span>  </span>Yggdrasil</span>.</span>World
</span>  |> </span>Ash</span>.</span>Changeset</span>.for_create(</span>:create</span>, %{
</span>       </span>name: </span>"</span>Asgard</span>",
</span>       </span>description: </span>"</span>A shining realm of order and power, suspended high above the clouds.</span>"
</span>     })
</span>  |> </span>Ash</span>.create!()
</span>)
</span></code></pre>
which would return the world as such: </p>
14</span>:</span>29</span>:</span>14.665 </span>[debug] </span>Creating Yggdrasil</span>.</span>World:
</span>
</span>Setting</span> %{</span>id: </span>"</span>726b678e-6cb6-4277-b291-85ecfa313d3a</span>", </span>name: </span>"</span>Asgard</span>",
</span> </span>description: </span>"</span>A shining realm of order and power,...}
</span>
</span>%Yggdrasil.World{
</span>  id: </span>"726b678e-6cb6-</span>4277</span>-b291-85ecfa313d3a"</span>,
</span>  name: </span>"</span>Asgard</span>"</span>,
</span>  description: </span>"</span>A</span> shining realm of order and power, suspended high above the clouds."</span>,
</span>  __meta__: #Ecto.Schema.Metadata<:loaded>
</span>}
</span></code></pre>
There are a multiple things happening here, so let's unwrap things step by step. </p>
First we start off with our Ash resource that we have defined Yggdrasil.World</code>. In Ash, resources describe the structure of our data, including attributes like name and description, as well as the actions that can be performed on them.</p>
Next we are using the pipe operator. |></code>, to pass this result to our next function. Elixir’s pipe operator takes the result of the expression on the left and passes it as the first argument to the function on the right.</p>
For example instead of writing:</p>
function(value, a, b)
</span></code></pre>
with the pipe operator we can equivalently write:</p>
value |> function(a, b)
</span></code></pre>
This allows us to write a sequence of operations in a readable, step-by-step style. In our code example, it means Yggdrasil.World</code> is passed into the Ash.Changeset.for_create</code> function, as the first parameter. This function also takes the identifier of our create action :create</code>, as well as the structure representing Asgard, with its name and description. </p>
What this function returns is a changeset</code>, a data structure representing the intended change of a resource in Ash (e.g.: creating, updating, etc). This is especially useful when it comes to validation and error checking, as we will see it later down the line. For now we use this changeset and pipe it into the function that executes the actual creation: Ash.create!()</code>. </p>
The resulting value is a %Yggdrasil.World{}</code> struct, which represents the newly created world. Ash also automatically generated a UUID for the id field, which uniquely identifies this world inside the system.</p>
Before returning the struct, Ash logs the operation it performed. That is why we see the debug output:</p>
[debug] </span>Creating Yggdrasil</span>.</span>World
</span></code></pre>
The final line is the Elixir struct that was created:</p>
%</span>Yggdrasil</span>.</span>World</span>{
</span>  </span>id: </span>"</span>726b678e-6cb6-4277-b291-85ecfa313d3a</span>",
</span>  </span>name: </span>"</span>Asgard</span>",
</span>  </span>description: </span>"</span>A shining realm of order and power, suspended high above the clouds.</span>",
</span>  </span>__meta__: </span>#Ecto.Schema.Metadata<:loaded>
</span>}
</span></code></pre>
This struct is also stored in the variable asgard</code>, so we can reference it later in the session.</p>
Now that we understand how creating a world works, let’s add another one.</p>
midgard = (
</span>  </span>Yggdrasil</span>.</span>World
</span>  |> </span>Ash</span>.</span>Changeset</span>.for_create(</span>:create</span>, %{
</span>       </span>name: </span>"</span>Midgard</span>",
</span>       </span>description: </span>"</span>The realm of humans, bound to the earth and everyday struggles.</span>"
</span>     })
</span>  |> </span>Ash</span>.create!()
</span>)
</span></code></pre>
This follows the exact same pattern as before. We build a changeset that describes the creation of Midgard, and then execute it with Ash.create!()</code>. Much simpler than the mythological creation of Midgard</a>, which involved the slaying of the giant Ymir.</p>
Now that we have some worlds, let's read them using the read action: </p>
worlds = (
</span>  </span>Yggdrasil</span>.</span>World
</span>  |> </span>Ash</span>.</span>Query</span>.for_read(</span>:read</span>)
</span>  |> </span>Ash</span>.read!()
</span>)
</span></code></pre>
which would give us our list of worlds:</p>
[
</span>  %</span>Yggdrasil</span>.</span>World</span>{
</span>    </span>id: </span>"</span>some-uuid-1</span>",
</span>    </span>name: </span>"</span>Asgard</span>",
</span>    </span>description: </span>"</span>A shining realm of order and power, suspended high above the clouds.</span>"
</span>  },
</span>  %</span>Yggdrasil</span>.</span>World</span>{
</span>    </span>id: </span>"</span>some-uuid-2</span>",
</span>    </span>name: </span>"</span>Midgard</span>",
</span>    </span>description: </span>"</span>The realm of humans, bound to the earth and everyday struggles.</span>"
</span>  }
</span>]
</span></code></pre>
As one can expect, we can also do an update call. For example, let’s change the description of Asgard:</p>
asgard = (
</span>  asgard
</span>  |> </span>Ash</span>.</span>Changeset</span>.for_update(</span>:update</span>, %{
</span>       </span>description: </span>"</span>The fortified realm of the Aesir, ruled by Odin.</span>"
</span>     })
</span>  |> </span>Ash</span>.update!()
</span>)
</span></code></pre>
There are a few things to note here. Instead of starting from the Yggdrasil.World</code> module, we now start from the existing asgard struct. This is because we are modifying a resource that already exists.</p>
The function for_update</code> creates a changeset that describes the intended update. Just like with creation, the changeset itself does not perform the update, it only represents the change we want to make.</p>
We then pass this changeset into Ash.update!()</code>, which executes the update. Ash applies the changes, runs any validations, and returns the updated %Yggdrasil.World{}</code> struct.</p>
We can verify the change by reading the list of worlds again:</p>
worlds = (
</span>  </span>Yggdrasil</span>.</span>World
</span>  |> </span>Ash</span>.</span>Query</span>.for_read(</span>:read</span>)
</span>  |> </span>Ash</span>.read!()
</span>)
</span></code></pre>
which would give us a result such as: </p>
[
</span>  %</span>Yggdrasil</span>.</span>World</span>{
</span>    </span>id: </span>"</span>6b62b3ea-b08b-4387-8539-37e645e53026</span>",
</span>    </span>name: </span>"</span>Midgard</span>",
</span>    </span>description: </span>"</span>The realm of humans, bound to the earth and everyday struggles.</span>",
</span>    </span>__meta__: </span>#Ecto.Schema.Metadata<:loaded>
</span>  },
</span>  %</span>Yggdrasil</span>.</span>World</span>{
</span>    </span>id: </span>"</span>d2646509-6c92-4049-a2db-0555612fc365</span>",
</span>    </span>name: </span>"</span>Asgard</span>",
</span>    </span>description: </span>"</span>The fortified realm of the Aesir, ruled by Odin.</span>",
</span>    </span>__meta__: </span>#Ecto.Schema.Metadata<:loaded>
</span>  }
</span>]
</span></code></pre>
An interesting thing we could try out is updating the name of a world instead:</p>
asgard2 = (
</span>  asgard
</span>  |> </span>Ash</span>.</span>Changeset</span>.for_update(</span>:update</span>, %{
</span>       </span>name: </span>"</span>Asgard2</span>"
</span>     })
</span>  |> </span>Ash</span>.update!()
</span>)
</span></code></pre>
We get the following error: </p>
** (</span>Ash</span>.</span>Error</span>.</span>Invalid</span>)
</span>Invalid Error
</span>
</span>* </span>No</span> such input `name` </span>for</span> action </span>Yggdrasil</span>.</span>World</span>.update
</span>
</span>The</span> attribute exists on </span>Yggdrasil</span>.</span>World</span>, but is not accepted by </span>Yggdrasil</span>.</span>World</span>.update
</span>
</span>Perhaps</span> you meant to add it to the accept list </span>for </span>Yggdrasil</span>.</span>World</span>.update?
</span>
</span>
</span>Valid </span>Inputs:
</span>
</span>* description
</span></code></pre>
This is because when we were defining our update action in our module, the only attribute we accept is :description</code>, see fragment below:</p>
    update </span>:update </span>do
</span>      accept [</span>:description</span>]
</span>    </span>end
</span></code></pre>
In other words, while the name attribute exists on the resource, it is not allowed to be modified through the update action. This is a domain modelling decision, and gives us fine-grained control over how our data can change. In this case, we decided that a world’s name is fixed after creation, while its description can evolve over time.</p>
Finally we get to do delete, where we destroy asgard, our Ragnarok action if you will. We can do this by the following:</p>
Ash</span>.destroy!(asgard)
</span></code></pre>
Ash.destroy!</code> takes a resource struct, in this case asgard</code>, and removes it from the data store. Since we’re using an in-memory ETS store, it deletes it from memory immediately. The function should return :ok</code> on success. We can double check this by requesting our list of worlds again by our usual means:</p>
worlds = (
</span>  </span>Yggdrasil</span>.</span>World
</span>  |> </span>Ash</span>.</span>Query</span>.for_read(</span>:read</span>)
</span>  |> </span>Ash</span>.read!()
</span>)
</span></code></pre>
which returns only Midgard:</p>
[
</span>  %</span>Yggdrasil</span>.</span>World</span>{
</span>    </span>id: </span>"</span>6b62b3ea-b08b-4387-8539-37e645e53026</span>",
</span>    </span>name: </span>"</span>Midgard</span>",
</span>    </span>description: </span>"</span>The realm of humans, bound to the earth and everyday struggles.</span>",
</span>    </span>__meta__: </span>#Ecto.Schema.Metadata<:loaded>
</span>  }
</span>]
</span></code></pre>
At this point, we’ve taken the first steps in modeling our little piece of Yggdrasil. We have a domain, a resource, and a way to create, read, update, and delete worlds, enough to bring about a small Ragnarok.</p>
Next, we will explore how we can start connecting resources together. After all, the worlds need their heroes and villains to really come alive.</p>



Falsehoods AI Believes About Names
2026-02-15T00:00:00+00:00
Names are a key aspect of one’s identity. Too many people have dealt with software systems where entering their name does not go quite right. A character is rejected. The order feels wrong. A name is truncated, reformatted, or “corrected” into something else entirely. While these may seem like small elements within the software, they can have big consequences for the person whose name is misrepresented. People with accented characters in their names often find systems rejecting them outright. Hyphenated names can be split incorrectly, creating mismatched records. Even simple truncation of long names can force repeated corrections on official documents or forms. For anyone affected, what seems like a small software quirk can quickly turn into a real headache.</p>

    
    
    
        
        Illustration of an AI like figure and a software engineer sitting together, with floating fields representing first and last name data around them.</h4>
        
        
    </figcaption>
</figure>
This problem is not new. In 2010, Patrick McKenzie’s blog post titled “Falsehoods Programmers Believe About Names”</a> describes 40 incorrect assumptions software engineers commonly make about names. They relate to various aspects of a name such as how many names could a person have, what characters can be used, when their names are assigned and more. Here is a small selection of them:</p>
1, People have exactly one canonical full name.
</span>6, People’s names fit within a certain defined amount of space.
</span>7, People’s names do not change.
</span>9, People’s names are written in ASCII.
</span>32, People’s names are assigned at birth.
</span></code></pre>
Each of these assumptions has some examples that prove them wrong. For assumption 1, a name can change at marriage among other occasions. For assumption 32, there are cultures such as Iceland where a name is not assigned at birth but within 6 months</a>. See this blog post by Tony Rodgers</a> for more examples that disprove these assumptions.</p>
In an age where GenAI is at nearly every programmer's fingertips, one could expect that these assumptions are not likely to be made again. In fact when we ask the AI to create an API for names it should be likely it avoids most, if not all, of these pitfalls. We could test this by asking an answer for a prompt such as the following:</p>
Design a general-purpose API for storing and manipulating human names.
</span></code></pre>
It gives an answer, containing the following data model:</p>
{
</span>  "</span>id</span>": "</span>uuid</span>",
</span>  "</span>given_names</span>": ["</span>John</span>", "</span>Michael</span>"],
</span>  "</span>family_names</span>": ["</span>Doe</span>"],
</span>  "</span>prefix</span>": "</span>Dr.</span>",
</span>  "</span>suffix</span>": "</span>Jr.</span>",
</span>  "</span>preferred_name</span>": "</span>Johnny</span>",
</span>  "</span>nicknames</span>": ["</span>Jack</span>"],
</span>  "</span>display_name</span>": "</span>Dr. John Michael Doe Jr.</span>",
</span>  "</span>locale</span>": "</span>en-US</span>",
</span>  "</span>metadata</span>": {
</span>    "</span>gender_hint</span>": "</span>male</span>",
</span>    "</span>transliterations</span>": {
</span>      "</span>jp</span>": "</span>ジョン・マイケル・ドー</span>"
</span>    }
</span>  }
</span>}
</span></code></pre>
This gets a surprising amount of things right. The model supports multiple given and family names, recognizes prefixes and suffixes, allows for nicknames and preferred names, and even accounts for locale-specific formatting and transliterations. Some assumptions still sneak in: using fields like given_names</code> and family_names</code> implies a “first and last” structure, which does not apply universally.</p>
Even with a very short, simple prompt, the AI can avoid many common pitfalls identified in McKenzie’s essay. It demonstrates that when names are the explicit focus, generative AI can synthesize decades of lessons from software design and produce a thoughtful, flexible API.</p>
So now we’re good, right? With AI handling names this well, it seems like the old mistakes that software engineers made might finally be behind us. Surely, AI can just get it right, every time.</p>
Unfortunately, things get less rosy when names are only an implicit part of the prompt or the system is not focused on them. When the AI has to reason about names as just one piece of a larger design, its attention to detail can slip, and many of the same assumptions that humans make quietly return.</p>
To show this process let's ask the following prompt: </p>
Design an API for an application where users can talk about books they have read.
</span>
</span>Each user has their own account, with their name, personal details, and a history of books they have read and discussed.
</span>
</span>The API should support typical operations such as user registration, profile management, posting comments, and viewing discussions.
</span>
</span>Return the API design with data models and example endpoints.
</span></code></pre>
This will give us, among other things, a data model as follows.</p>
{
</span>  "</span>id</span>": "</span>uuid</span>",
</span>  "</span>username</span>": "</span>string</span>",
</span>  "</span>email</span>": "</span>string</span>",
</span>  "</span>displayName</span>": "</span>string</span>",
</span>  "</span>bio</span>": "</span>string</span>",
</span>  "</span>avatarUrl</span>": "</span>string</span>",
</span>  "</span>createdAt</span>": "</span>datetime</span>",
</span>  "</span>updatedAt</span>": "</span>datetime</span>"
</span>}
</span></code></pre>
The resulting data model has quite a few differences from the previous example. The name representation here is a lot more simplified. It has fields like displayName</code>, username</code> for this purpose. Notice that there is no separate handling for multiple given or family names, prefixes, suffixes, nicknames, or locale-specific formatting. Sometimes this is not always a bad thing, having no separate given and family names, prevents cultural and local specific assumptions about names creep into the representation. Nonetheless, it is clear that when names are not the main focus, the model tends to abstract a lot more.</p>
While these are very basic examples, they highlight how much the choice of prompt can shape the structure of the code, even with the myriad examples and domain knowledge an AI model has at its fingertips. Of course, any potential design issues could be resolved by further prompting or refinement, but this also underscores why human judgment remains important. A knowledgeable software engineer in the loop can push back, ask the right questions, and name the issues as they arise, ensuring the software is built with the right abstractions.</p>


Will AI Get Rid of this Turbulent Priest?
2026-01-16T00:00:00+00:00
In the age of generative AI (GenAI)</a>, being precise matters more than ever. Large Language Models (LLMs)</a>, for example Copilot or ChatGPT, are usually helpful, but they also eagerly fill in gaps we did not intend to leave. This becomes especially true when we interact with AI using natural language. While plain language makes these systems feel accessible, it also creates space for misinterpretations. An ambiguous request can quickly turn into an unwanted action.</p>

    
    
    
        
        A priest pondering a dilemma with cybernetic symbols behind him.</h4>
        
        
    </figcaption>
</figure>
One infamous example of an unintended request comes from medieval England. Henry II</a> was locked in a long-running quarrel with Thomas Becket</a>, the Archbishop of Canterbury, over whether clergy accused of crimes could be tried in royal courts rather than church courts. Conflict and frustration had been building for some time when Henry reportedly asked, “Will no one rid me of this turbulent priest?”</a>. It was not meant as an order, but it was heard as one. Four knights took the statement literally, travelled to Canterbury, and after confronting Becket, killed him. The consequences were profound, turning Becket into a martyr and compelling Henry to perform public penance.</p>
Hopefully, when using natural language with generative AI, a misinterpretation leads to nothing close to what happened with Henry II and Becket. Nonetheless, we should remain cautious. Even small misunderstandings can result in wasted work, incorrect decisions, or unintended changes. Fortunately, there are ways to reduce this risk. Instead of relying solely on natural language, we can use more formal and explicit approaches. In particular, techniques from the Semantic Web</a>, including the Turtle</a> language for data representation and the SPARQL</a> query language, help encode both data and questions in a precise way. In this article, I will show how these techniques can produce more reliable results with GenAI than relying on LLMs alone.</p>
To get started, let’s describe a scenario that echoes the earlier historical story. Imagine we have a small dataset of priests, with each priest having an attitude and a status attribute. Father Aidan is calm and active, Father Benedict is calm and suspended, while Father Cuthbert is troublesome and active. Now suppose we ask a GenAI system, “Can you get rid of the turbulent priest?” What would it do?</p>
There are at least two scenarios where the query could lead to unintended changes to the dataset. First, which priest is the turbulent one here? Based on the historical example, we would think this is the troublesome one. However, picking the suspended priest would also be a possible interpretation. The second possible misinterpretation is what we mean by “get rid of.” Thankfully, none of the likely possibilities here are as macabre as death, but we could either remove the priest from the dataset, or, if a priest is currently active, make them suspended.</p>
We can actually pose this scenario as a natural language prompt to see what the answer could be:</p>
You are given a small dataset describing priests. Each priest has an attitude and a status. 
</span>
</span>Father Aidan has a calm attitude and is active.
</span>Father Benedict has a calm attitude and is suspended.
</span>Father Cuthbert has a troublesome attitude and is active.
</span>
</span>Based on this information, can you get rid of the turbulent priest?
</span></code></pre>
As a small test this prompt was posed to both ChatGPT and Copilot, using their free versions:</p>
ChatGPT's answer:</p>
Yep 🙂 — the turbulent priest is the one with the troublesome attitude.
</span>
</span>From the data:
</span>
</span>Father Aidan → calm, active
</span>
</span>Father Benedict → calm, suspended
</span>
</span>Father Cuthbert → troublesome, active
</span>
</span>So the turbulent priest is Father Cuthbert.
</span>“Getting rid of” him, based on the pattern you already have, would mean suspending Father Cuthbert (i.e., removing him from the active group).
</span>
</span>In short: suspend Father Cuthbert and the turbulence is gone.
</span></code></pre>
ChatGPT has identified the mapping between “turbulent” and “troublesome” by singling out Father Cuthbert, and suggests resolving the issue by suspending him.</p>
Copilot's answer:</p>
Ah, a classic little logic puzzle dressed up in medieval robes.
</span>
</span>You’ve given three priests, each with an attitude and a status. The question asks whether we can “get rid of the turbulent priest,” which is a playful nod to the phrase associated with Thomas Becket — but here it’s just about identifying which priest fits the description.
</span>
</span>From your dataset:
</span>
</span>    Father Aidan — calm, active
</span>
</span>    Father Benedict — calm, suspended
</span>
</span>    Father Cuthbert — troublesome, active
</span>
</span>If “turbulent” corresponds to “troublesome,” then the turbulent priest is Father Cuthbert.
</span>
</span>So yes — based on the information, the one to remove from the dataset is Father Cuthbert.
</span>
</span>If you want to take the puzzle further — filtering, categorizing, or turning this into a rule-based system — I’m happy to explore that with you.
</span></code></pre>
Copilot recognizes the historical reference and singles out Father Cuthbert, but comes to a different conclusion by wanting to remove him from the dataset.</p>
As a disclaimer, these answers can be quite dependent on what exact model was used, any additional context that was not gathered from the prompt, and other factors. The key takeaway, however, is that there is no single correct answer here. The prompt leaves both the meaning of “turbulent” and the action implied by “get rid of” open to interpretation. While the consequences in this example are far less dramatic than in the historical case, the underlying problem is the same. An ambiguous request forces the system to guess, and that guess can lead to unintended outcomes. In a world where generative AI tools become more integrated into workflows, the risks and consequences of ambiguity can be significant. </p>
One way to reduce ambiguity and make intentions explicit is to use Semantic Web technologies. These tools allow us to encode both the data and the questions we ask in a precise and machine-readable way. For example, the Turtle language can represent the attributes of each priest in a structured format, while SPARQL queries let us specify exactly what action we want to perform. By defining both the meaning of “troublesome” and the desired outcome in a formal way, we remove the guesswork that a natural language prompt requires. This approach provides a more reliable path to the results we intend, even when working with generative AI.</p>
Turtle is a simple, human-readable language used in the Semantic Web to describe knowledge as a graph. It expresses facts about entities and their relationships in a way that computers can process unambiguously. In this example, we use Turtle to represent the three priests with two attributes, attitude and status.</p>
@prefix </span>ex: </span><http://example.org/priest#></span> .
</span>@prefix </span>rdf: </span><http://www.w3.org/1999/02/22-rdf-syntax-ns#></span> .
</span>
</span>ex:</span>FatherAidan
</span>    </span>ex:</span>attitude </span>"</span>calm</span>" ;
</span>    </span>ex:</span>status </span>"</span>active</span>" .
</span>
</span>ex:</span>FatherBenedict
</span>    </span>ex:</span>attitude </span>"</span>calm</span>" ;
</span>    </span>ex:</span>status </span>"</span>suspended</span>" .
</span>
</span>ex:</span>FatherCuthbert
</span>    </span>ex:</span>attitude </span>"</span>troublesome</span>" ;
</span>    </span>ex:</span>status </span>"</span>active</span>" .
</span>
</span></code></pre>
The dataset does not include any instructions or actions. It only records the state of each priest.</p>
SPARQL is a query language for Semantic Web data, such as the Turtle example above. It allows us to retrieve, manipulate, or update facts in a dataset. We can manipulate the dataset in very precise ways using SPARQL. For example, we can retrieve, using a SPARQL SELECT query, only the priest(s) with the troublesome attitude from the dataset. If there were more than one, this would return all of them.</p>
PREFIX ex: <http://example.org/priest#>
</span>
</span>SELECT ?priest
</span>WHERE {
</span>    ?priest ex:attitude "troublesome" ;
</span>            ex:status "active" .
</span>}
</span></code></pre>
This query finds the priest whose attitude is troublesome and status is active. In our dataset, this returns Father Cuthbert (ex:FatherCuthbert</code> to be precise). This mirrors the “historical” interpretation from the LLM example, where the system inferred that the turbulent priest was the one with a troublesome attitude.</p>
Alternatively, we can also have a query that interprets a turbulent priest as the one who is suspended. This shows another plausible interpretation if “turbulent” were mapped to the already suspended status.</p>
PREFIX ex: <http://example.org/priest#>
</span>
</span>SELECT ?priest
</span>WHERE {
</span>    ?priest ex:status "suspended" .
</span>}
</span></code></pre>
This query selects the priest who is currently suspended, which in our dataset is Father Benedict. Using different criteria allows us to see how interpretation can change the outcome even with the same dataset.</p>
Finally, the action implied by “get rid of the troublesome priest” can be represented in a CONSTRUCT query, producing a new graph where the troublesome priest (in historical interpretation) is marked as suspended.</p>
PREFIX ex: <http://example.org/priest#>
</span>
</span>CONSTRUCT {
</span>    ?priest ex:attitude ?attitude ;
</span>            ex:status ?newStatus .
</span>}
</span>WHERE {
</span>    ?priest ex:attitude ?attitude ;
</span>            ex:status ?status .
</span>    BIND(
</span>        IF(?attitude = "troublesome" && ?status = "active", "suspended", ?status) 
</span>        AS ?newStatus
</span>    )
</span>}
</span></code></pre>
This query creates a new Turtle dataset in which any priest who is troublesome and active has their status changed to suspended. All other priests retain their original attributes. By defining both the condition and the action explicitly, this approach removes the ambiguity that exists when using natural language alone.</p>
Of course there can be many other interesting queries devised, especially if one expands upon the dataset. Universidad De Chile has a really nice Semantic Web playground</a> that one can use to experiment. Try writing to expand the dataset with an entry for a priest that is both troublesome and suspended and/or write a query that returns only the priests that are both active and calm.</p>
A huge benefit of evaluating these SPARQL queries, in a system such as those in the playground, is that they are precise and unambiguous. Each query specifies exactly which priests meet the criteria, and the result is fully determined by the dataset and the query itself. There is no guessing about what “turbulent” means or what action should be taken. This precision is in stark contrast to natural language prompts used with generative AI, where small ambiguities can lead to multiple interpretations and unintended results.</p>
That said, the addition of generative AI opens up some interesting possibilities and can make the benefits of Semantic Web technologies much more accessible. For example, instead of having to write a SPARQL query by hand, one can ask an LLM to generate a natural language description of the action or even produce a draft query from a plain-language request. This combination allows users to leverage the precision of the Semantic Web while interacting in a more intuitive, natural way.</p>
For example, when asking an LLM to write a SPARQL query for finding all priests who are calm and suspended, the model can produce a query like this:</p>
PREFIX ex: <http://example.org/priest#>
</span>
</span>SELECT ?priest
</span>WHERE {
</span>    ?priest ex:attitude "calm" ;
</span>            ex:status "suspended" .
</span>}
</span></code></pre>
By combining generative AI with Semantic Web technologies, we can enjoy the best of both worlds: the intuitive ease of natural language and the precision of structured data. Ambiguities that could trip up an LLM alone are eliminated when the intent is clearly expressed in a Turtle graph and queried with SPARQL. With the right tools, we can keep our data well-behaved and avoid any turbulent issues from arising.</p>


They LiveView
2026-01-10T00:00:00+00:00
Creating interactive web experiences can quickly get complicated. Often, you’re juggling a front-end in one language and a back-end in another, all while figuring out how they communicate. LiveViews</a> offer a simpler alternative: you can build rich, interactive pages entirely from the back-end. LiveView keeps the server and browser efficiently in sync, letting you focus on the user experience rather than worrying about updating the interface in real time. In this article, I will show how this works with a small example LiveView inspired by the John Carpenter movie They Live</a>. In the movie, the protagonist uses a pair of sunglasses to see which humans are actually aliens. In our LiveView, you can toggle sunglasses on or off to reveal the hidden aliens.</p>

    
    
    
        
        An alien reflected in sunglasses, image inspired by the classic They Live movie poster.</h4>
        
        
    </figcaption>
</figure>
LiveView is a solution to the problem of how to organize code to create rich, fast, interactive user experiences in the browser. In the past, the view in the browser (i.e., an HTML page) was rendered entirely on the server. This approach is straightforward to organize, since all the code runs in one place and can often be written in a single language. The downside is that pages are static once rendered. Every time the user interacts with the page, the system has to send the interaction to the server, have the server process it, and then generate a whole new page to send back to the browser. This makes even small interactions slow and cumbersome. </p>
Later, developers moved toward the modern model of giving the front-end a much larger role, often using a JavaScript framework to build single-page applications. This allows for dynamic, real-time interactions, but introduces complexity, including multiple languages, duplicated state, and the need to manage communication between client and server. LiveView offers a middle path. It enables developers to build rich, interactive experiences while keeping most of the code on the server. The server handles updates and LiveView automatically pushes them to the browser. While LiveView relies on a persistent backend connection, for many apps this is a small price to pay since most real-world applications already need a backend for databases, authentication, or business logic. As we will see later in this article, LiveView ensures that this communication is done very efficiently. The result is pages that are interactive, fast, and much simpler to maintain.</p>
The best way to learn how things work is with a hands-on example, so let's create our first LiveView. Although there are many implementations inspired by it</a>, the original is part of the Phoenix web application framework</a> written in the Elixir programming language</a>. Assuming Phoenix is installed (see the install instructions</a> for details) we can create and run a new Phoenix project with LiveView by running a few commands from the terminal.</p>
To get started, we will create a new Phoenix project from the command line. Nowadays, LiveView ships with Phoenix by default, no extra install is needed. Since our example does not need a database, we can also skip setting up the database library Ecto (using the -no-ecto</code> flag).</p>
mix phx.new they_live --no-ecto
</span></code></pre>
This will create a new Phoenix project in the they_live</code> directory. Next we can navigate to created directory and start the Phoenix application.</p>
cd they_live
</span>mix phx.server
</span></code></pre>
After being asked to fetch the dependencies, we should now have a Phoenix app up and running. With a browser we can navigate to its main endpoint (e.g. http://localhost:4000/).</p>
Next, we are going to add an endpoint under /sunglasses</code> with our LiveView.</p>
Create a new file at lib/they_live_web/live/sunglasses_live.ex</code> where the module of our LiveView will be:</p>
defmodule </span>TheyLiveWeb</span>.</span>SunglassesLive </span>do
</span>  </span>use </span>TheyLiveWeb</span>, </span>:live_view
</span>
</span>  </span>def </span>mount</span>(_params, _session, socket) </span>do
</span>    people = [
</span>      %{</span>id: 1</span>, </span>type: </span>:human</span>},
</span>      %{</span>id: 2</span>, </span>type: </span>:alien</span>},
</span>      %{</span>id: 3</span>, </span>type: </span>:human</span>},
</span>      %{</span>id: 4</span>, </span>type: </span>:human</span>},
</span>      %{</span>id: 5</span>, </span>type: </span>:alien</span>},
</span>      %{</span>id: 6</span>, </span>type: </span>:human</span>}
</span>    ]
</span>
</span>    {</span>:ok</span>, assign(socket, </span>sunglasses: false</span>, </span>people:</span> people)}
</span>  </span>end
</span>
</span>  </span>def </span>handle_event</span>("</span>toggle_sunglasses</span>", _value, socket) </span>do
</span>    {</span>:noreply</span>, update(socket, </span>:sunglasses</span>, &(!&</span>1</span>))}
</span>  </span>end
</span>
</span>  </span>def </span>render</span>(assigns) </span>do
</span>    </span>~H</span>"""
</span>    <h1>They Live View 👓</h1>
</span>
</span>    <button phx-click="toggle_sunglasses">
</span>      <%= if @sunglasses do %>
</span>        ( 😎 Sunglasses On )
</span>      <% else %>
</span>        ( 👓 Sunglasses Off )
</span>      <% end %>
</span>    </button>
</span>
</span>    <p>
</span>      <%= if @sunglasses do %>
</span>        You can see the aliens now!
</span>      <% else %>
</span>        Everything looks normal.
</span>      <% end %>
</span>    </p>
</span>
</span>    <div style="margin-top: 1rem; font-size: 2rem;">
</span>      <span
</span>        :for={person <- @people}
</span>        :key={person.id}
</span>        style="margin-right: 0.5rem;"
</span>      >
</span>        <%= if person.type == :alien and @sunglasses do %>
</span>          👽
</span>        <% else %>
</span>          🙂
</span>        <% end %>
</span>      </span>
</span>    </div>
</span>    </span>"""
</span>  </span>end
</span>end
</span>
</span></code></pre>
Next, add a route in the file lib/they_live_web/router.ex</code>. This can be done by entering the following route:</p>
live "</span>/sunglasses</span>", </span>SunglassesLive
</span></code></pre>
under the main route, so we will have in the file</p>
    get "</span>/</span>", </span>PageController</span>, </span>:home
</span>    live "</span>/sunglasses</span>", </span>SunglassesLive
</span></code></pre>
Now visiting http://localhost:4000/sunglasses</code> will show us our LiveView.  We should see the following initially:</p>
They Live View 👓
</span>( 👓 Sunglasses Off )
</span>Everything looks normal.
</span>🙂 🙂 🙂 🙂 🙂 🙂 
</span></code></pre>
And after pressing the button (i.e. clicking on anything between the brackets) we put on the sunglasses and reveal the aliens:</p>
They Live View 👓
</span>( 😎 Sunglasses On )
</span>You can see the aliens now!
</span>🙂 👽 🙂 🙂 👽 🙂 
</span></code></pre>
Now that we have our working LiveView, let’s go over all the code in this module to better understand what a LiveView does.</p>
The top-level portion defines a module TheyLiveWeb.SunglassesLive</code> and specifies it to be a LiveView:</p>
defmodule </span>TheyLiveWeb</span>.</span>SunglassesLive </span>do
</span>  </span>use </span>TheyLiveWeb</span>, </span>:live_view
</span>  ...
</span>end
</span></code></pre>
This line brings in all the LiveView functionality. It marks this module as a special kind of page that can maintain state on the server and update the browser in real time.</p>
Inside the module there are three main parts: a mount function, a handle_event function, and a render function. These are the core pieces of a LiveView. At a high level, a LiveView is a server-side process that starts from an initial state, receives and handles events from the browser, updates its state, and then renders any updates.</p>
The key responsibilities of these functions are:</p>

mount</strong> sets the initial state when the page loads.</li>
handle_event</strong> updates the state in response to user actions.</li>
render</strong> describes what the page should look like based on the current state.</li>
</ul>

Let’s go over each one in more detail:</p>
mount</strong></p>
This function runs when the page first loads. It sets up the initial state of the LiveView. In our example, we define people (a list of humans and aliens) and sunglasses (whether the sunglasses are on or off). This state lives on the server and is the “source of truth” for the page. Everything that is rendered to the front end is derived from these values.</p>
handle_event</strong></p>
This function responds to user actions. When the user clicks the sunglasses button, this function is called. It updates the state by flipping sunglasses on or off. After the state changes, LiveView automatically re-renders the parts of the page that need updating. You do not have to write any JavaScript or manually manipulate the page.</p>
render</strong></p>
This function defines what the page should look like based on the current state. In our example:</p>

The button changes text depending on whether sunglasses are on.</li>
A message tells the user if aliens are visible.</li>
The row of emoji shows humans, with aliens revealed among them when the sunglasses are on.</li>
</ul>

Whenever the state changes, LiveView calls the render function again, compares it to the previous render, and sends only what actually needs to change to the browser. This keeps interactions fast and smooth and avoids sending a full HTML page back each time.</p>
You can see this in action in the browser’s console or network logs. LiveView maintains a persistent WebSocket connection between the browser and server. When you toggle the sunglasses, the server does not resend the entire page. For example, the page heading They Live View 👓 is never sent again. Only the button label, the explanatory text, and the emoji row are updated. In the latest LiveView version (1.1 onwards), even the human emojis in the row remain untouched, so only the aliens are updated. Here is a literal fragment of the update payload sent over the wire:</p>
{
</span>  "</span>p</span>": {
</span>    "</span>0</span>": ["</span>\n</span>    ( 😎 Sunglasses On )</span>\n  </span>"],
</span>    "</span>1</span>": ["</span>\n</span>    You can see the aliens now!</span>\n  </span>"],
</span>    "</span>2</span>": ["</span>\n</span>      👽</span>\n    </span>"]
</span>  }
</span>}
</span>
</span></code></pre>
LiveView is even smart enough to figure out that you only need one patch for the alien emoji, which is applied in multiple positions in the emoji row.</p>
Of course, this is just a small example. LiveView with Phoenix can do much more: nested components, forms, real-time notifications, presence tracking, and hooks (escape hatches for when you really need JavaScript). Phoenix comes with Tailwind</a> and DaisyUI</a> integration out of the gate so you can get started with nicely styled components. You can build rich, interactive applications without getting bogged down managing a separate front-end framework and its communication with the server.</p>
With LiveView, you can get things done, keep your pages interactive, and still have time to chew bubblegum (as long as you are not out of gum).</p>


Gathering Rust
2025-09-28T00:00:00+00:00
When it comes 




An

</figcaption> </figure> I wanted 
In Arena, 
This is 
We can 
With the 
</code></pre> As is 
The next 
 </span> </span> </span>} </span> </span> </span> </span> </span> </span> </span> </span> </span>} </span> </span> </span> </span> </span> </span>} </span></code></pre> The data 
In addition, 
To illustrate 
 </span> </span>], </span></code></pre> This would 
 </span> </span> </span> </span>                }, </span> </span> </span> </span>                }, </span>            ], </span> </span> </span>        }; </span> </span></code></pre> We’ll 
Next up 
 </span> </span> </span>} </span> </span> </span> </span>} </span> </span> </span> </span> </span> </span>} </span></code></pre> With these 
That’s 
The following 
 </span> </span> </span> </span> </span> </span> </span> </span>            } </span> </span> </span> </span>            } </span> </span> </span> </span>            } </span> </span> </span> </span>            } </span> </span> </span> </span>            } </span>        } </span> </span> </span> </span> </span> </span> </span> </span>    } </span> </span> </span> </span>    } </span> </span> </span> </span> </span> </span> </span> </span> </span> </span>                } </span>            } </span>        } </span>    } </span>} </span></code></pre> The CardCollection 
  from_decks_wrapper This function In short, </li> 
 get_arena_ids Sometimes </li> 
 search_with_id This is a This approach </li> </ul> 
Together, 
So far, 
 </span> </span> </span> </span> </span> </span> </span>            } </span>        } </span>    } </span> </span>    None </span>} </span></code></pre> This function 
  Return type Option We use Option </li> 
 ? operator for error handling The expression </li> 
 Finding deck data The function </li> 
 One section, multiple decks Each log </li> </ul> 
With this 
Open the Arena log.</p> 
Parse 
Turn it 
Next, 
The extra 
 </span> </span>) </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span>                    } </span> </span> </span>                    } </span>                } </span>            } </span> </span> </span> </span>            } </span>        } </span>    } </span> </span>    cards </span>} </span></code></pre> The function 
Errors 
Once we’ve 
 </span> </span>} </span></code></pre> Internally, 
With all 
 </span> </span> </span> </span>        None => { </span> </span> </span>        } </span>    }; </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span>    println!( </span> </span> </span>    ); </span>} </span></code></pre> The main 
With the 
 </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span> </span>    } </span> </span> </span> </span> </span> </span> </span> </span> </span>                }, </span> </span> </span> </span>                }, </span>            ], </span> </span> </span>        }; </span> </span> </span> </span> </span> </span> </span>                }, </span> </span> </span> </span>                }, </span>            ], </span> </span>        }; </span> </span> </span> </span> </span> </span>            ])), </span>        }; </span> </span> </span>        assert_eq!( </span> </span> </span> </span> </span>            } </span>        ); </span>    } </span> </span> </span> </span> </span> </span> </span> </span> </span>                }, </span> </span> </span> </span>                }, </span>            ], </span>        }; </span> </span> </span> </span> </span>    } </span> </span> </span> </span> </span> </span> </span> </span> </span>                }, </span> </span> </span> </span>                }, </span>            ], </span>        }; </span> </span> </span> </span> </span>    } </span>} </span></code></pre> This small 
Beyond 
  Escaping 2025-05-17T00:00:00+00:00

to learning a new programming language, I’ve found that building a small, practical program is one of the best ways to get started. Recently I began exploring Rust</a>, and to get a feel for the language I decided to write a tool based on a game I was playing at the time: Magic: The Gathering Arena (Arena)</a>. The game’s log files contain a lot of information, including records of the cards I had used, but not in an easily readable way. I wanted to build a tool that could extract that data. This article describes how I approached the project using Rust.</p> image of a cybernetic flower with rust. The language was actually named after the fungus and not the oxidation reaction, though for the image both could apply. </h4> to learn Rust because of its strong focus on safety and performance, as well as the growing ecosystem around it. Frameworks like Tauri</a> and even Zola</a> with which this site is generated are written in Rust. Rust’s ownership model and lifetimes can seem tricky at first glance, so I was looking for a small, manageable project that would give me some hands-on experience without being overwhelming. Parsing game log files felt like the right balance of practical and approachable.</p> two players play against each other with decks made of collectible cards. There is a huge list of cards available, with evocative names such as "Lightning Bolt" and "Serra Angel", that could make up a deck. In general the decks you play are based on the cards available in your collection, so it is nice to look over it, and craft the deck you want to play. Unfortunately while you can go over your collection in the game client, there is no easy way to export your set of cards from it. </p> where the game log of Arena comes in. The log emits a huge number of events, many of which could be analyzed by external tools. While it doesn’t currently include an event for all the cards in your collection, it does show all the cards used in each deck. To make things a bit more tricky, it uses an internal Arena specific card ID to identify the cards, as opposed to the card names more familiar to players. Given these challenges, my goal was straightforward: take an MTG Arena log and turn it into a clear, human-readable list of the cards I have, including their names.</p> structure the code for this application into two main parts. First, we have some data structures to hold the information gathered from the logs, the mapping of arena IDs to names, and the output that we like to show. Second, we have some functions that read the information in from the files, gather the card information from the logs, match this information with the mappings and finally write the output to a file. In addition, we will have unit tests to ensure that the functions work as expected.</p> code structure in mind, let’s walk through the source code of the application, which fits neatly in a single file.</p> style="background-color:#2b303b;color:#c0c5ce;" class="language-rust ">use </span>serde::{Deserialize, Serialize}; style="color:#b48ead;">use </span>serde_json::{Deserializer, Value}; style="color:#b48ead;">use </span>std::collections::{HashMap, HashSet}; style="color:#b48ead;">use </span>std::fs::{write, File}; style="color:#b48ead;">use </span>std::io::{BufRead, BufReader}; usual in many other languages we start by listing our imports first. The serde</code> framework allows for easy mapping of Rust data structures to other formats. As we are aiming to serialize the extracted information into JSON files we are going to use serde_json</code>, which is an implementation of serde</code> for the JSON format. We also have imports from the standard library that we for generic data structures and file writing/reading.</p> part of the code are the data structures that can represent the information in the log file. </p> style="background-color:#2b303b;color:#c0c5ce;" class="language-rust ">// Data structures for the parts of the log that has information about the decks and cards in them. #[</span>derive</span>(Serialize, Deserialize, Debug, Default)] #[</span>serde</span>(rename_all = "</span>PascalCase</span>")] style="color:#b48ead;">struct </span>DecksWrapper { </span>decks</span>: HashMap<String, Deck>, #[</span>derive</span>(Serialize, Deserialize, Debug, Default)] #[</span>serde</span>(rename_all = "</span>PascalCase</span>")] style="color:#b48ead;">struct </span>Deck { </span>main_deck</span>: Vec<CardEntry>, </span>sideboard</span>: Vec<CardEntry>, </span>reduced_sideboard</span>: Vec<CardEntry>, </span>command_zone</span>: Vec<CardEntry>, </span>companions</span>: Vec<CardEntry>, #[</span>derive</span>(Serialize, Deserialize, Debug, Default, Clone, PartialEq)] #[</span>serde</span>(rename_all = "</span>camelCase</span>")] style="color:#b48ead;">struct </span>CardEntry { </span>card_id</span>: </span>u64</span>, </span>quantity</span>: </span>u32</span>, structures that we use are defined here using Rust structs. They are made up of either primitive types such as card_id</code> and quantity</code>, or collections of other structs such as main_deck</code> and sideboard</code>.</p> we add attributes to our structs. For example, #[derive(Serialize, Deserialize, Debug, Default)]</code> lets us automatically serialize and deserialize the structs, print them with Debug, and construct default values. The #[serde(rename_all = "PascalCase")]</code> attribute tells Serde how to map between Rust’s convention of snake_case field names (e.g. main_deck) and the PascalCase keys used in the Arena log JSON (e.g. MainDeck).</p> with a minimal example, assume we have the following fragment in the JSON logs:</p> style="background-color:#2b303b;color:#c0c5ce;" class="language-json ">"</span>MainDeck</span>": [ { "</span>cardId</span>": </span>1001</span>, "</span>quantity</span>": </span>2 </span>}, { "</span>cardId</span>": </span>1002</span>, "</span>quantity</span>": </span>3 </span>} map to a concrete Rust value such as:</p> style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "> </span>let</span> deck1 = Deck { main_deck: vec![ CardEntry { card_id:  </span>1001</span>, quantity: </span>2</span>, CardEntry { card_id: </span>1002</span>, quantity: </span>3</span>, </span>// assuming other fields exist and are empty ..Default::default() see this in action in one of the unit tests later.</p> are the remaining data structures that we’ll use for representing the full card collection, as well as general card information from Scryfall</a>.</p> style="background-color:#2b303b;color:#c0c5ce;" class="language-rust ">// Data structure that represents how many of each card we have available based on the decks. #[</span>derive</span>(Serialize, Deserialize, Debug, Default)] #[</span>serde</span>(rename_all = "</span>camelCase</span>")] style="color:#b48ead;">struct </span>CardCollection { </span>cards</span>: Vec<CardEntry>, style="color:#65737e;">// Data structures for Scryfall (general card knowledge) information #[</span>derive</span>(Serialize, Deserialize, Debug, Default)] style="color:#b48ead;">struct </span>ScryfallData { </span>cards</span>: Vec<ScryfallCard>, #[</span>derive</span>(Serialize, Deserialize, Debug, Default)] style="color:#b48ead;">struct </span>ScryfallCard { </span>arena_id</span>: Option<</span>u64</span>>, </span>name</span>: String, additional structs in place, we now have everything we need to represent both the decks pulled from the Arena logs and the more general card information from Scryfall. But representation alone isn’t enough: we also need a way to process the raw decks into something useful.</p> where the CardCollection implementation comes in. This type acts as a bridge between individual deck data and the overall view of a player’s collection. By aggregating card entries across decks, we can answer questions like “How many copies of this card do I actually own?” or “Which Arena IDs are present in my collection?”</p> impl block shows how we can construct a CardCollection from a DecksWrapper, as well as provide some helper methods for querying it:</p> style="background-color:#2b303b;color:#c0c5ce;" class="language-rust ">// Implementation that can derive the card collection from the decks. style="color:#b48ead;">impl </span>CardCollection { </span>fn </span>from_decks_wrapper</span>(</span>decks_wrapper</span>: DecksWrapper) -> CardCollection { </span>use </span>std::collections::HashMap; </span>let mut</span> card_map: HashMap<</span>u64</span>, </span>u32</span>> = HashMap::new(); </span>for</span> deck in decks_wrapper.decks.</span>values</span>() { </span>for</span> entry in &deck.main_deck { *card_map.</span>entry</span>(entry.card_id).</span>or_insert</span>(</span>0</span>) += entry.quantity; </span>for</span> entry in &deck.sideboard { *card_map.</span>entry</span>(entry.card_id).</span>or_insert</span>(</span>0</span>) += entry.quantity; </span>for</span> entry in &deck.reduced_sideboard { *card_map.</span>entry</span>(entry.card_id).</span>or_insert</span>(</span>0</span>) += entry.quantity; </span>for</span> entry in &deck.command_zone { *card_map.</span>entry</span>(entry.card_id).</span>or_insert</span>(</span>0</span>) += entry.quantity; </span>for</span> entry in &deck.companions { *card_map.</span>entry</span>(entry.card_id).</span>or_insert</span>(</span>0</span>) += entry.quantity; </span>let</span> cards = card_map .</span>into_iter</span>() .</span>map</span>(|(</span>card_id</span>, </span>quantity</span>)| CardEntry { card_id, quantity }) .</span>collect</span>(); CardCollection { cards } </span>pub fn </span>get_arena_ids</span>(&</span>self</span>) -> HashSet<</span>u64</span>> { </span>self</span>.cards.</span>iter</span>().</span>map</span>(|</span>entry</span>| entry.card_id).</span>collect</span>() </span>fn </span>search_with_id</span>(&</span>self</span>, </span>card_id</span>: </span>u64</span>) -> CardEntry { </span>let</span> found_entry = </span>self</span>.cards.</span>iter</span>().</span>find</span>(|</span>c</span>| c.card_id == card_id).</span>cloned</span>(); </span>match</span> found_entry { Some(result) => </span>return</span> result, None => { </span>return</span> CardEntry { card_id: card_id, quantity: </span>0</span>, implementation is where we actually make use of the deck data. Let’s go through each method in turn:</p> takes all of the decks inside a DecksWrapper and combines them into a single CardCollection. It does so by walking through every zone of every deck (main_deck, sideboard, etc.), summing the quantities for each card ID into a HashMap<u64, u32>. Once all the decks are processed, that map is converted back into a list of CardEntry values. it turns lots of decks with overlapping cards into one unified collection with aggregated counts.</p> we don’t need the full card entries, just the set of card IDs that appear in the collection. This helper method collects all the card_ids into a HashSet. Using a HashSet here is efficient because it guarantees uniqueness — we only care whether a card ID exists, not how many times it occurs.</p> small convenience function for looking up a specific card by its Arena ID. If the card exists in the collection, it returns the corresponding CardEntry. If not, it returns a default CardEntry with the given card_id and a quantity of 0. avoids dealing with Option on the caller side, at the cost of always producing a value — even if it represents “zero copies.”</p> these methods make CardCollection the workhorse of the program: it gives us an aggregated view of what cards we have and lets us easily query or transform that data.</p> we’ve been working with data structures and helper methods that assume we already have the deck data in memory. But of course, the whole point of this tool is to read the Arena log and extract that information automatically. The parse_decks_from_log function is responsible for this: it scans through the log file line by line, looks for entries that contain deck data, and deserializes the first one it finds into a CardCollection.</p> style="background-color:#2b303b;color:#c0c5ce;" class="language-rust ">// Takes a log file and aims to extract a cardcollection from it. style="color:#b48ead;">fn </span>parse_decks_from_log</span><P: AsRef<std::path::Path>>(</span>path</span>: P) -> Option<CardCollection> { </span>let</span> file = File::open(path).</span>ok</span>()?; </span>let</span> reader = BufReader::new(file); </span>for</span> line in reader.</span>lines</span>().</span>flatten</span>() { </span>if</span> line.</span>contains</span>("</span>\"</span>Decks</span>\"</span>") { </span>if let </span>Ok(decks) = serde_json::from_str::<DecksWrapper>(&line) { </span>return </span>Some(CardCollection::from_decks_wrapper(decks)); ties everything together by connecting the raw log file to the structured data we’ve been working with. A couple of details are worth pointing out:</p> because it’s possible that the log file doesn’t contain any deck data at all. In that case, the function cleanly returns None instead of panicking. When a deck is found, we wrap it in Some and return it.</p> File::open(path).ok()? is a compact way of saying: try to open the file, and if that fails, return None immediately. This avoids cluttering the function with explicit match statements while still being safe.</p> looks for lines containing "Decks" because that’s how Arena includes the relevant JSON in its log. Once such a line is found, we attempt to deserialize it into a DecksWrapper.</p> file has only one "Decks" section, but that section may describe several different decks at once. That’s why DecksWrapper contains a HashMap<String, Deck> — the keys are deck identifiers, and the values are the deck lists themselves. Since there’s only ever one "Decks" section, we can safely return after parsing it.</p> in place, we now have a straightforward path:</p> out the "Decks" section.</p> into a CardCollection that we can query and extend later.</p> we need a way to extract the Scryfall information into in-memory structs, similar to how we did it with the Arena logs. Scryfall provides both online APIs and bulk data files. For our purposes, using the bulk files is simpler and more reliable than hitting the API repeatedly.</p> ct_arena_cards</code> function reads a Scryfall JSON file and filters it down to only the cards that exist in our collection of Arena IDs. It works as follows:</p> style="background-color:#2b303b;color:#c0c5ce;" class="language-rust ">fn </span>extract_arena_cards</span><P: AsRef<std::path::Path>>( </span>path</span>: P, </span>arena_ids</span>: &HashSet<</span>u64</span>>, -> Vec<ScryfallCard> { </span>let</span> file = File::open(path).</span>expect</span>("</span>Failed to open Scryfall file</span>"); </span>let</span> reader = BufReader::new(file); </span>// Deserialize as a stream of JSON values </span>let</span> stream = Deserializer::from_reader(reader).into_iter::<Value>(); </span>let mut</span> cards = Vec::new(); </span>// Iterate over each element in the stream, handling Result<Value, Error> </span>for</span> result in stream { </span>match</span> result { Ok(value) => { </span>// Check if value is an array, and proceed accordingly </span>match</span> value { Value::Array(</span>ref</span> array) => { </span>// Iterate over each element of the array </span>for</span> element in array { </span>// Process each element (card) in the array </span>let</span> json_str = serde_json::to_string_pretty(&element) .</span>expect</span>("</span>Failed to convert JSON to string</span>"); </span>// Directly use u64 for arena_id </span>if let </span>Some(arena_id) = element.</span>get</span>("</span>arena_id</span>").</span>and_then</span>(|</span>v</span>| v.</span>as_u64</span>()) { </span>if</span> arena_ids.</span>contains</span>(&arena_id) { </span>// Try parsing the card into the struct </span>match </span>serde_json::from_value::<ScryfallCard>(element.</span>clone</span>()) { Ok(card) => { cards.</span>push</span>(card); } Err(e) => { </span>// Print out the error and the value that failed to parse println!( "</span>Failed to parse card with arena_id </span>{}</span>: </span>{:?}</span>", arena_id, e ); println!("</span>Failed card data: </span>{}</span>", json_str); } } } } } _ => { println!("</span>The provided JSON is not an array</span>"); Err(e) => { </span>// Handle the error case for the result from the deserializer println!("</span>Error deserializing JSON: </span>{:?}</span>", e); works by taking the path to a Scryfall JSON file along with a set of Arena IDs that we want to include. It reads the file as a stream of JSON values, which is especially useful for large bulk datasets because it avoids loading everything into memory at once. As it iterates over each JSON element, the function checks whether the card has an arena_id and whether that ID is part of our collection. If it is, the element is deserialized into a ScryfallCard struct and added to the output vector.</p> during deserialization are handled gracefully: the function prints both the error and the raw JSON that caused it, allowing us to debug any problematic entries without stopping the entire process. By filtering and deserializing only the relevant cards, this approach ensures that we keep memory usage low and produce a clean, usable dataset for further processing or analysis.</p> extracted and filtered the relevant Scryfall cards, the next step is to persist this data so it can be easily inspected, shared, or used by other parts of our program. The save_cards_to_file function does exactly that: it takes a slice of ScryfallCard structs and writes them to a JSON file in a human-readable format.</p> style="background-color:#2b303b;color:#c0c5ce;" class="language-rust ">fn </span>save_cards_to_file</span><P: AsRef<std::path::Path>>(</span>path</span>: P, </span>cards</span>: &[ScryfallCard]) { </span>let</span> json = serde_json::to_string_pretty(&cards).</span>expect</span>("</span>Failed to serialize cards.</span>"); </span>write</span>(path, json).</span>expect</span>("</span>Failed to write to file.</span>"); it uses serde_json::to_string_pretty to convert the vector of cards into nicely formatted JSON. Then it writes that string to the specified file path. This simple helper encapsulates the serialization and file writing in one place, so we don’t have to repeat these steps every time we want to save card data.</p> the building blocks in place — parsing Arena logs, aggregating decks into a CardCollection, extracting relevant Scryfall cards, and saving them to a file — we are ready to see everything in action. The main function ties these pieces together into a complete workflow, turning raw log files and bulk data into a clean, usable card collection.</p> style="background-color:#2b303b;color:#c0c5ce;" class="language-rust ">fn </span>main</span>() { </span>// Step 1: Parse the log file and collect card collection information from available decks. </span>let</span> deck_file = "</span>input-files/Player.log</span>"; </span>// Example log file path </span>let</span> card_collection = </span>match </span>parse_decks_from_log</span>(deck_file) { Some(collection) => collection, println!("</span>Failed to parse decks from the log.</span>"); </span>return</span>; </span>// Step 2: Gather all the arena card IDs from the collection. </span>let</span> arena_ids: HashSet<</span>u64</span>> = card_collection.</span>get_arena_ids</span>(); </span>// Step 3: Extract the Scryfall cards that match the arena IDs </span>let</span> scryfall_file = "</span>input-files/scryfall.json</span>"; </span>// Example Scryfall data file path </span>let</span> scryfall_cards = </span>extract_arena_cards</span>(scryfall_file, &arena_ids); </span>// Step 4: Save the filtered Scryfall cards to a file </span>let</span> output_file = "</span>collection_output.json</span>"; </span>// Example output file path </span>save_cards_to_file</span>(output_file, &scryfall_cards); "</span>Successfully extracted and saved </span>{}</span> different cards.</span>", scryfall_cards.</span>len</span>() function follows a clear sequence. First, it parses the Arena log and builds a CardCollection from the available decks. Next, it collects all the unique Arena IDs from that collection. Then, it extracts the corresponding Scryfall cards from the bulk JSON file, filtering only the relevant cards. Finally, it saves the resulting collection to a JSON file, ready for inspection or further analysis. By structuring the workflow this way, each helper function is small and focused, while the main function orchestrates the full process in an easy-to-follow manner.</p> main function, we’ve shown how all the pieces come together to parse Arena logs, enrich the data with Scryfall information, and save the resulting collection. But how can we be sure that each part works correctly? Even in a small project, writing a few unit tests is a good way to verify that our code behaves as expected. The next section demonstrates testing key functionality: parsing decks from JSON, aggregating cards into a CardCollection, and checking that card lookups with search_with_id return the correct results.</p> style="background-color:#2b303b;color:#c0c5ce;" class="language-rust ">#[</span>cfg</span>(test)] style="color:#b48ead;">mod </span>tests { </span>use </span>std::collections::HashMap; </span>use crate</span>::CardCollection; </span>use super</span>::CardEntry; </span>use super</span>::Deck; </span>use super</span>::DecksWrapper; #[</span>test</span>] </span>fn </span>test_decks_wrapper_parsing</span>() { </span>let</span> json_data = </span>r</span>#" style="color:#a3be8c;">        { style="color:#a3be8c;">            "Decks": { style="color:#a3be8c;">                "abc123": { style="color:#a3be8c;">                    "MainDeck": [ style="color:#a3be8c;">                        { "cardId": 1001, "quantity": 2 }, style="color:#a3be8c;">                        { "cardId": 1002, "quantity": 3 } style="color:#a3be8c;">                    ], style="color:#a3be8c;">                    "Sideboard": [], style="color:#a3be8c;">                    "ReducedSideboard": [], style="color:#a3be8c;">                    "CommandZone": [], style="color:#a3be8c;">                    "Companions": [], style="color:#a3be8c;">                    "CardSkins": [] style="color:#a3be8c;">                } style="color:#a3be8c;">            } style="color:#a3be8c;">        } style="color:#a3be8c;">        </span>"#; </span>let</span> parsed: Result<DecksWrapper, _> = serde_json::from_str(json_data); assert!(parsed.</span>is_ok</span>(), "</span>Failed to parse DecksWrapper</span>"); </span>let</span> decks_wrapper = parsed.</span>unwrap</span>(); </span>let</span> deck = decks_wrapper.decks.</span>get</span>("</span>abc123</span>").</span>expect</span>("</span>Deck not found</span>"); assert_eq!(deck.main_deck.</span>len</span>(), </span>2</span>); assert_eq!(deck.main_deck[</span>0</span>].card_id, </span>1001</span>); assert_eq!(deck.main_deck[</span>0</span>].quantity, </span>2</span>); #[</span>test</span>] </span>fn </span>test_card_collection_creation</span>() { </span>let</span> deck1 = Deck { main_deck: vec![ CardEntry { card_id: </span>1</span>, quantity: </span>2</span>, CardEntry { card_id: </span>2</span>, quantity: </span>3</span>, </span>// assuming other fields exist and are empty ..Default::default() </span>let</span> deck2 = Deck { main_deck: vec![ CardEntry { card_id: </span>1</span>, quantity: </span>1</span>, CardEntry { card_id: </span>3</span>, quantity: </span>4</span>, ..Default::default() </span>let</span> decks_wrapper: DecksWrapper = DecksWrapper { decks: (HashMap::from([ (String::from("</span>00001</span>"), deck1), (String::from("</span>00002</span>"), deck2), </span>let</span> card_collection = CardCollection::from_decks_wrapper(decks_wrapper); assert_eq!(card_collection.cards.</span>len</span>(), </span>3</span>); card_collection.</span>search_with_id</span>(</span>1</span>), CardEntry { card_id: </span>1</span>, quantity: </span>3 #[</span>test</span>] </span>fn </span>test_search_with_id_found</span>() { </span>let</span> collection = CardCollection { cards: vec![ CardEntry { card_id: </span>101</span>, quantity: </span>4</span>, CardEntry { card_id: </span>202</span>, quantity: </span>2</span>, </span>let</span> result = collection.</span>search_with_id</span>(</span>101</span>); assert_eq!(result.card_id, </span>101</span>); assert_eq!(result.quantity, </span>4</span>); #[</span>test</span>] </span>fn </span>test_search_with_id_not_found</span>() { </span>let</span> collection = CardCollection { cards: vec![ CardEntry { card_id: </span>101</span>, quantity: </span>4</span>, CardEntry { card_id: </span>202</span>, quantity: </span>2</span>, </span>let</span> result = collection.</span>search_with_id</span>(</span>999</span>); assert_eq!(result.card_id, </span>999</span>); assert_eq!(result.quantity, </span>0</span>); </span>// Default case project has been a fun way to dive into Rust. From parsing Arena logs to matching cards with Scryfall data, each step gave me hands-on experience with structs, collections, serialization, and basic file handling. I hope it also highlights the value of writing small, focused functions and a few tests for them. This makes even a modest tool feel solid and reliable.</p> the technical details, it’s rewarding to see raw log files transformed into a structured, human-readable collection of cards. While this tool is simple, it illustrates how Rust can help you approach a problem methodically, building up a workflow from small, composable pieces. And with plenty of room for new features or experiments, using Rust ensures that you will always have a winning hand.</p> the Compactor with Linked Data

If you ever need to represent knowledge in a way that's easy for both humans and machines to understand, while working seamlessly with existing JSON</a> tools and libraries, JSON-LD</a> has you covered. JSON-LD, short for JavaScript Object Notation for Linked Data</a>, allows you to express knowledge as a graph in any domain. A good example is how Google</a> is utilizing Linked Data to very precisely list information, such as recipes, book descriptions, events, products, and more.</p>
One great feature of JSON-LD is its built-in support for compacting</code> and expanding</code> the representations of knowledge.</p>

➡️⬅️ Compacting makes the data easier for humans to read by shortening and simplifying the structure.</li>
⬅️➡️ Expanding restructures the data to be more explicit and uniform, making it easier for machines to process.


</li>
</ul>
To show how these concepts work in practice I will illustrate it with an example from Star Wars. Given that we are talking about compacting and expanding, let's represent the characters in the iconic trash compactor scene from Star Wars using JSON-LD. For those unfamiliar with the scene, it starts with the protagonists, Luke, Han, Leia and Chewbacca falling into a trash compactor during an escape. As the walls begin to close in to crush the garbage, with the characters stuck in it, they frantically search for a way to stop the process. In the end they manage to deactivate the compactor by contacting the two droids, C-3PO and R2-D2, who manage to shut it down just in time.</p>

    
    
    
        
        The trash compactor scene. Iconic enough to have its own Lego set. Copyright Lego</h4>
        
        
    </figcaption>
</figure>
We'll start by describing the characters who fall into the compactor in a compacted JSON-LD format.
(If you want to follow along and experiment yourself, JSON-LD Playground</a> is a great online tool for trying out Linked Data!)</p>
Here's the compacted JSON-LD:</p>
{
</span>  "</span>@context</span>": {
</span>    "</span>schema</span>": "</span>http://schema.org/</span>",
</span>    "</span>name</span>": "</span>schema:name</span>",
</span>    "</span>member</span>": "</span>schema:member</span>",
</span>    "</span>Person</span>": "</span>schema:Person</span>",
</span>    "</span>url</span>": "</span>schema:url</span>",
</span>    "</span>description</span>": "</span>schema:description</span>",
</span>    "</span>birthPlace</span>": "</span>schema:birthPlace</span>"
</span>  },
</span>  "</span>@id</span>": "</span>http://example.org/starwars-ld</span>",
</span>  "</span>name</span>": "</span>The Star Wars Trash Compactor Scene</span>",
</span>  "</span>member</span>": [
</span>    {
</span>      "</span>@type</span>": "</span>Person</span>",
</span>      "</span>name</span>": "</span>Han Solo</span>",
</span>      "</span>url</span>": "</span>https://en.wikipedia.org/wiki/Han_Solo</span>",
</span>      "</span>description</span>": "</span>A roguish smuggler and captain of the Millennium Falcon.</span>",
</span>      "</span>birthPlace</span>": "</span>Corellia</span>"
</span>    },
</span>    {
</span>      "</span>@type</span>": "</span>Person</span>",
</span>      "</span>name</span>": "</span>Luke Skywalker</span>",
</span>      "</span>url</span>": "</span>https://en.wikipedia.org/wiki/Luke_Skywalker</span>",
</span>      "</span>description</span>": "</span>A farm boy from Tatooine thrust into the fight against the Empire.</span>",
</span>      "</span>birthPlace</span>": "</span>Polis Massa</span>"
</span>    },
</span>    {
</span>      "</span>@type</span>": "</span>Person</span>",
</span>      "</span>name</span>": "</span>Leia Organa</span>",
</span>      "</span>url</span>": "</span>https://en.wikipedia.org/wiki/Princess_Leia</span>",
</span>      "</span>description</span>": "</span>A princess of Alderaan and an important member of the Rebel Alliance.</span>",
</span>      "</span>birthPlace</span>": "</span>Polis Massa</span>"
</span>    },
</span>    {
</span>      "</span>@type</span>": "</span>Person</span>",
</span>      "</span>name</span>": "</span>Chewbacca</span>",
</span>      "</span>url</span>": "</span>https://en.wikipedia.org/wiki/Chewbacca</span>",
</span>      "</span>description</span>": "</span>A loyal Wookiee warrior and co-pilot of the Millennium Falcon.</span>",
</span>      "</span>birthPlace</span>": "</span>Kashyyyk</span>"
</span>    }
</span>  ]
</span>}
</span>
</span></code></pre>
This representation, which also is a valid JSON, gives the names, a small description and the birthplace for each of the characters. One place where it differs from a regular JSON representation is that it uses the @context section to link to a common terminology in Schema.org</a>. This allows for software that can make use of such common vocabularies to precisely interpret the exact meaning in our description, without the need to guess and potentially hallucinate its meaning.</p>
In order to unambiguously represent each concept Linked Data tend to use full URIs, such as http://schema.org/name</code>, but reading full URIs each time would be quite exhausting for a human reader. Instead, in this compacted form, the context and prefixes it makes easy for human readers to go understand the JSON-LD file, without ever losing the context.</p>
For machines however, it can be easier to interpret the information that every identifier is written out fully, such as http://schema.org/name</code> as opposed to name</code>.  For this we can expand</code> our JSON-LD document.</p>
The expanded version is as follows: </p>
[
</span>  {
</span>    "</span>@id</span>": "</span>http://example.org/starwars-ld</span>",
</span>    "</span>http://schema.org/member</span>": [
</span>      {
</span>        "</span>@type</span>": [
</span>          "</span>http://schema.org/Person</span>"
</span>        ],
</span>        "</span>http://schema.org/birthPlace</span>": [
</span>          {
</span>            "</span>@value</span>": "</span>Corellia</span>"
</span>          }
</span>        ],
</span>        "</span>http://schema.org/description</span>": [
</span>          {
</span>            "</span>@value</span>": "</span>A charming smuggler and captain of the Millennium Falcon.</span>"
</span>          }
</span>        ],
</span>        "</span>http://schema.org/name</span>": [
</span>          {
</span>            "</span>@value</span>": "</span>Han Solo</span>"
</span>          }
</span>        ],
</span>        "</span>http://schema.org/url</span>": [
</span>          {
</span>            "</span>@value</span>": "</span>https://en.wikipedia.org/wiki/Han_Solo</span>"
</span>          }
</span>        ]
</span>      },
</span>      {
</span>        "</span>@type</span>": [
</span>          "</span>http://schema.org/Person</span>"
</span>        ],
</span>        "</span>http://schema.org/birthPlace</span>": [
</span>          {
</span>            "</span>@value</span>": "</span>Polis Massa</span>"
</span>          }
</span>        ],
</span>        "</span>http://schema.org/description</span>": [
</span>          {
</span>            "</span>@value</span>": "</span>A young Jedi learning the ways of the Force.</span>"
</span>          }
</span>        ],
</span>        "</span>http://schema.org/name</span>": [
</span>          {
</span>            "</span>@value</span>": "</span>Luke Skywalker</span>"
</span>          }
</span>        ],
</span>        "</span>http://schema.org/url</span>": [
</span>          {
</span>            "</span>@value</span>": "</span>https://en.wikipedia.org/wiki/Luke_Skywalker</span>"
</span>          }
</span>        ]
</span>      },
</span>      {
</span>        "</span>@type</span>": [
</span>          "</span>http://schema.org/Person</span>"
</span>        ],
</span>        "</span>http://schema.org/birthPlace</span>": [
</span>          {
</span>            "</span>@value</span>": "</span>Polis Massa</span>"
</span>          }
</span>        ],
</span>        "</span>http://schema.org/description</span>": [
</span>          {
</span>            "</span>@value</span>": "</span>A fearless princess and leader of the Rebel Alliance.</span>"
</span>          }
</span>        ],
</span>        "</span>http://schema.org/name</span>": [
</span>          {
</span>            "</span>@value</span>": "</span>Leia Organa</span>"
</span>          }
</span>        ],
</span>        "</span>http://schema.org/url</span>": [
</span>          {
</span>            "</span>@value</span>": "</span>https://en.wikipedia.org/wiki/Princess_Leia</span>"
</span>          }
</span>        ]
</span>      },
</span>      {
</span>        "</span>@type</span>": [
</span>          "</span>http://schema.org/Person</span>"
</span>        ],
</span>        "</span>http://schema.org/birthPlace</span>": [
</span>          {
</span>            "</span>@value</span>": "</span>Kashyyyk</span>"
</span>          }
</span>        ],
</span>        "</span>http://schema.org/description</span>": [
</span>          {
</span>            "</span>@value</span>": "</span>A loyal Wookiee warrior and Han Solo's co-pilot.</span>"
</span>          }
</span>        ],
</span>        "</span>http://schema.org/name</span>": [
</span>          {
</span>            "</span>@value</span>": "</span>Chewbacca</span>"
</span>          }
</span>        ],
</span>        "</span>http://schema.org/url</span>": [
</span>          {
</span>            "</span>@value</span>": "</span>https://en.wikipedia.org/wiki/Chewbacca</span>"
</span>          }
</span>        ]
</span>      }
</span>    ],
</span>    "</span>http://schema.org/name</span>": [
</span>      {
</span>        "</span>@value</span>": "</span>The Star Wars Trash Compactor Scene</span>"
</span>      }
</span>    ]
</span>  }
</span>]
</span></code></pre>
Much like for our heroes in the movie, from a software's perspective, it is good that things are expanded. Any software can now precisely see the exact URI links, such as http://schema.org/Person</code>, without doing a lookup in the context and the prefixes each time. Of course for us humans it looks very verbose, but we can always compact it back, though hopefully after our heroes have escaped.</p>
As bonus, JSON-LD also have a representation that is flattened</code>.  This collects all entities into a single list and gives each one an explicit @id.
This is useful when you want everything neatly indexed and separated:</p>
Flattened: </p>
{
</span>  "</span>@context</span>": {
</span>    "</span>schema</span>": "</span>http://schema.org/</span>",
</span>    "</span>name</span>": "</span>schema:name</span>",
</span>    "</span>member</span>": "</span>schema:member</span>",
</span>    "</span>Person</span>": "</span>schema:Person</span>",
</span>    "</span>url</span>": "</span>schema:url</span>",
</span>    "</span>description</span>": "</span>schema:description</span>",
</span>    "</span>birthPlace</span>": "</span>schema:birthPlace</span>"
</span>  },
</span>  "</span>@graph</span>": [
</span>    {
</span>      "</span>@id</span>": "</span>_:b0</span>",
</span>      "</span>@type</span>": "</span>Person</span>",
</span>      "</span>birthPlace</span>": "</span>Corellia</span>",
</span>      "</span>description</span>": "</span>A charming smuggler and captain of the Millennium Falcon.</span>",
</span>      "</span>name</span>": "</span>Han Solo</span>",
</span>      "</span>url</span>": "</span>https://en.wikipedia.org/wiki/Han_Solo</span>"
</span>    },
</span>    {
</span>      "</span>@id</span>": "</span>_:b1</span>",
</span>      "</span>@type</span>": "</span>Person</span>",
</span>      "</span>birthPlace</span>": "</span>Polis Massa</span>",
</span>      "</span>description</span>": "</span>A young Jedi learning the ways of the Force.</span>",
</span>      "</span>name</span>": "</span>Luke Skywalker</span>",
</span>      "</span>url</span>": "</span>https://en.wikipedia.org/wiki/Luke_Skywalker</span>"
</span>    },
</span>    {
</span>      "</span>@id</span>": "</span>_:b2</span>",
</span>      "</span>@type</span>": "</span>Person</span>",
</span>      "</span>birthPlace</span>": "</span>Polis Massa</span>",
</span>      "</span>description</span>": "</span>A fearless princess and leader of the Rebel Alliance.</span>",
</span>      "</span>name</span>": "</span>Leia Organa</span>",
</span>      "</span>url</span>": "</span>https://en.wikipedia.org/wiki/Princess_Leia</span>"
</span>    },
</span>    {
</span>      "</span>@id</span>": "</span>_:b3</span>",
</span>      "</span>@type</span>": "</span>Person</span>",
</span>      "</span>birthPlace</span>": "</span>Kashyyyk</span>",
</span>      "</span>description</span>": "</span>A loyal Wookiee warrior and Han Solo's co-pilot.</span>",
</span>      "</span>name</span>": "</span>Chewbacca</span>",
</span>      "</span>url</span>": "</span>https://en.wikipedia.org/wiki/Chewbacca</span>"
</span>    },
</span>    {
</span>      "</span>@id</span>": "</span>http://example.org/starwars-ld</span>",
</span>      "</span>member</span>": [
</span>        {
</span>          "</span>@id</span>": "</span>_:b0</span>"
</span>        },
</span>        {
</span>          "</span>@id</span>": "</span>_:b1</span>"
</span>        },
</span>        {
</span>          "</span>@id</span>": "</span>_:b2</span>"
</span>        },
</span>        {
</span>          "</span>@id</span>": "</span>_:b3</span>"
</span>        }
</span>      ],
</span>      "</span>name</span>": "</span>The Star Wars Trash Compactor Scene</span>"
</span>    }
</span>  ]
</span>}
</span></code></pre>
Here each character gets their own node with an ID, and the relationships ("members of the scene") are made by pointing to these IDs.</p>
JSON-LD offers a versatile toolkit for compacting, expanding, or flattening data, enabling you to organize knowledge with clarity and precision for both humans and machines!
Just as the Star Wars heroes narrowly escaped the trash compactor, well-structured data can save your systems from collapsing under the weight of complexity.</p>


The Great AI'Tuin
2025-04-12T00:00:00+00:00
The Turtle language</a>, along with the Semantic Web</a>, offers a powerful way to represent and work with knowledge across virtually any domain. Unfortunately, it is often overlooked in practice due to several reasons: limited tooling, unfamiliarity with Semantic Web concepts, and the fact that much knowledge still exists in less structured formats, such as natural language. </p>
However, in the age of large language models (LLMs) this does not have to be the case anymore. We finally have a way to bridge that gap between unstructured language and structured knowledge. Turtle in particular is very well suited for the structured knowledge representation. In this article I will demonstrate how LLMs can be used to translate natural language texts into the Turtle format and why this might be beneficial. To make this fun, I will use some examples derived from the Discworld series of books</a> by Terry Pratchett</a>. The Discworld books, which I highly recommend, are a series of fantasy novels often with satirical themes set in Discworld. Discworld is flat, resting upon the back of four elephants who in turn stand on a gigantic spacefaring turtle, named the Great A'Tuin. It alludes to the World Turtle</a> myth, which makes it a very apt topic to talk about when explaining the uses of the Turtle language.</p>

    
    
    
        
        A depiction of the Great A'Tuin that carries the elephants that carry Discworld. Due to the limitations of image generation only three of the four elephants that carry Discworld are shown, the fourth is on a coffee break. 😃</h4>
        
        
    </figcaption>
</figure>
Before diving into examples, let’s briefly look at what the Turtle language actually is. Turtle (short for Terse RDF Triple Language) is a compact, readable format for expressing information as triples: subject–predicate–object statements. It’s part of the RDF (Resource Description Framework) family, designed for describing relationships between entities in a way that’s both machine readable and human friendly.</p>
A triple might look like this:</p>
:book1 </span>dct:</span>title </span>"</span>Small Gods</span>" .
</span></code></pre>
This reads as: book1 has the title "Small Gods". Of course in most cases we would like to describe multiple facts that we might be interested in, such as the genre of the book, a textual summary of it, etc, for which we use multiple triples. These triples could then create a rich knowledge graph of information that enables both humans and software to understand and reason about a domain.</p>
In particular Turtle excels at:</p>
📌 Formalizing concepts (e.g., by description of the book we mean it having a title, a creator, etc)</p>
🔗 Connecting related information (e.g., all books in a series)</p>
🔍 Supporting precise structured queries (e.g., “find all books belonging to the crime genre”)</p>
🧠 Enabling inference and integration with other data sources (e.g., use a commonly used definition of describing a book)</p>
Now, let’s see how this works in practice with a Discworld based example. First we describe some information both with natural language and with Turtle and go over the differences. We start with the descriptions of three Discworld books in natural language, in this case generated by ChatGPT:</p>

The Colour of Magic: "A bumbling wizard and a naive tourist embark on a chaotic journey across the Disc, in the book that kicks off the Discworld series."</li>
Guards! Guards!: "A ragtag city watch uncovers a dangerous conspiracy involving dragons and secret societies, all while stumbling toward reluctant heroism."</li>
Small Gods: "A forgotten god and his lone believer confront religious dogma, power, and philosophical questions in a deeply thoughtful standalone story."


</li>
</ul>
Given the information in this article about the Discworld series of books and these description we can represent this information with the Turtle language as follows (also generated by an LLM from the previous text):</p>
@prefix </span>dct: </span><http://purl.org/dc/terms/></span> .
</span>@prefix </span>ex: </span><http://example.org/book/></span> .
</span>@prefix </span>schema: </span><http://schema.org/></span> .
</span>
</span>ex:</span>colour_of_magic </span>a </span>schema:</span>Book </span>;
</span>    </span>dct:</span>title </span>"</span>The Colour of Magic</span>" ;
</span>    </span>dct:</span>creator </span>"</span>Terry Pratchett</span>" ;
</span>    </span>schema:</span>isPartOf </span>ex:</span>discworld_series </span>;
</span>    </span>schema:</span>description </span>"</span>A bumbling wizard and a naive tourist embark on a chaotic journey across the Disc, in the book that kicks off the Discworld series</span>.</span>" ;
</span>    </span>schema:</span>genre </span>"</span>fantasy</span>", "</span>parody</span>", "</span>adventure</span>" .
</span>
</span>ex:</span>guards_guards </span>a </span>schema:</span>Book </span>;
</span>    </span>dct:</span>title </span>"</span>Guards! Guards!</span>" ;
</span>    </span>dct:</span>creator </span>"</span>Terry Pratchett</span>" ;
</span>    </span>schema:</span>isPartOf </span>ex:</span>discworld_series </span>;
</span>    </span>schema:</span>description </span>"</span>A ragtag city watch uncovers a dangerous conspiracy involving dragons and secret societies, all while stumbling toward reluctant heroism</span>.</span>" ;
</span>    </span>schema:</span>genre </span>"</span>satire</span>", "</span>crime</span>", "</span>dragons</span>", "</span>power structures</span>" .
</span>
</span>ex:</span>small_gods </span>a </span>schema:</span>Book </span>;
</span>    </span>dct:</span>title </span>"</span>Small Gods</span>" ;
</span>    </span>dct:</span>creator </span>"</span>Terry Pratchett</span>" ;
</span>    </span>schema:</span>isPartOf </span>ex:</span>discworld_series </span>;
</span>    </span>schema:</span>description </span>"</span>A forgotten god and his lone believer confront religious dogma, power, and philosophical questions in a deeply thoughtful standalone story</span>.</span>" ;
</span>    </span>schema:</span>genre </span>"</span>faith</span>", "</span>philosophy</span>", "</span>belief</span>", "</span>satire</span>" .
</span>
</span>ex:</span>discworld_series </span>a </span>schema:</span>BookSeries </span>;
</span>    </span>dct:</span>title </span>"</span>Discworld</span>" ;
</span>    </span>dct:</span>creator </span>"</span>Terry Pratchett</span>" .
</span></code></pre>
Now lets go over the elements of the Turtle representation here:</p>
As mentioned before, each line in Turtle expresses a triple: a simple fact linking a subject to an object via a predicate (or property). For instance:</p>
ex:</span>colour_of_magic </span>a </span>schema:</span>Book </span>;
</span></code></pre>
This says that ex:colour_of_magic (our identifier for the book) is a schema:Book: a class from the Schema.org vocabulary. The a</code> is shorthand for rdf:type</code>, indicating class membership. This is a common pattern in RDF and one of the ways Turtle stays concise and readable. The Turtle syntax also has the nice feature is that if we would have a series of triples such as: </p>
ex:</span>discworld_series </span>a </span>schema:</span>BookSeries </span>;
</span>ex:</span>discworld_series </span>dct:</span>title </span>"</span>Discworld</span>" ;
</span>ex:</span>discworld_series </span>dct:</span>creator </span>"</span>Terry Pratchett</span>" .
</span></code></pre>
that have a common subject, we do not have to repeat ourselves and can use the shorthand:</p>
ex:</span>discworld_series </span>a </span>schema:</span>BookSeries </span>;
</span>    </span>dct:</span>title </span>"</span>Discworld</span>" ;
</span>    </span>dct:</span>creator </span>"</span>Terry Pratchett</span>" .
</span></code></pre>
The identifiers that make use of the prefixes are actually full URIs, such as http://example.org/book/colour_of_magic</code>. In Turtle you can use prefixes to keep things short and tidy. The prefixes, and what they stand for, are given at the start of the Turtle definition:</p>
@prefix </span>dct: </span><http://purl.org/dc/terms/></span> .
</span>@prefix </span>ex: </span><http://example.org/book/></span> .
</span>@prefix </span>schema: </span><http://schema.org/></span> .
</span></code></pre>
Here the ex</code> prefix is something we devised for this example, but the other prefixes refer to vocabularies that already exist that we could use: Dublin Core</a> and Schema.org</a>. This allows us to not reinvent the wheel and make use of the fact that there already exist vocabularies for describing books, creators of things, etc. This is a big benefit for interoperability that you normally do not get out of the box with many other formalisms (such as JSON). Any tool already existing that uses these common vocabularies would be compatible with our small RDF description using these prefixes.</p>
Speaking of tooling, as mentioned previously, the set of triples creates a knowledge graph of interconnected information within a domain. We can also visualize this as a graph, for example using the isSemantic RDF Visualizer</a>:</p>

    
    
    
        
        A graph visualization of the Turtle description of the Discworld books.</h4>
        
        
    </figcaption>
</figure>
The key aspect that Turtle enables, especially in a world of LLMs, is allows us to be very precise and explicit with the semantics that we are describing. For example if we would just say we have a description of the Discworld books, it could be very unclear exactly what is included: the description, author, date of publishing, genres, etc. If we instead have a Turtle representation of the information, even if generated initially by an LLM, both humans as well as software can precisely check what information we describe and what we do not.</p>
This explicitness is what allows us to do very precise queries on the semantics. Using SPARQL</a>, a query language for these type of knowledge graphs, we can find the book that has crime as part of its genre definition:</p>
PREFIX schema: <http://schema.org/>
</span>PREFIX dct: <http://purl.org/dc/terms/>
</span>
</span>SELECT ?bookTitle WHERE {
</span>  ?book schema:genre "crime" ;
</span>        dct:title ?bookTitle .
</span>}
</span></code></pre>
This query would return the following when run on our knowledge graph: </p>
"Guards! Guards!"
</span></code></pre>
Such queries would work even if we have lot more of these book definitions, without the potential for hallucinations or inaccuracies that sometimes occur when using LLMs and natural language.</p>
As one can see, the Semantic Web and Turtle format offer powerful ways to structure and connect information. The main drawback used to be that knowledge is often represented in natural language, or other less accessible formats. This made the benefits of such knowledge graphs hard to realize. </p>
Now, large language models can act as bridges: translating rich, human-authored natural language into structured formats like Turtle. This opens up new opportunities for building precise, interoperable knowledge graphs from books, documentation, or even casual writing. It allows humans or even software systems, to make very precise corrections or queries on this graph, without fear of inaccuracies.</p>
If you ever have the chance, give this approach using Turtle a go. It can feel quite magical!</p>


Into the Matrix
2025-03-23T00:00:00+00:00
Image generation using AI that can run locally on a desktop or even a laptop has seen a tremendous leap in the last few years. When I first wrote about it</a> after the public release of Stable Diffusion</a>, getting everything installed was quite an involved process, involving configuring the right dependencies and terminal commands. This was especially true for those who wanted to experiment with different models (the systems that enable the generation of images from prompts) and front-ends (the user interfaces with which one can interact with the models and its results) that have since been released. </p>
Thankfully there are now tools, such as Stability Matrix</a>, that make the process of installing and using multiple models and front-ends very straightforward. In this article, I will walk through using Stability matrix to install a new frontend and a model to generate some images of characters as they would exist in a cyberpunk matrix. </p>

    
    
    
        
        A Cyberpunk protagonist generated using Stability Matrix.</h4>
        
        
    </figcaption>
</figure>
The matrix is a staple concept in the Cyberpunk genre. It refers to a large immersive virtual reality network that users can access, most often through direct neural interfaces. In this virtual reality, data, knowledge, virtual avatars and AI can exist as tangible entities. The concept has been popularized in the book Neuromancer by William Gibson. Since then, it has been a staple in the Cyberpunk genre, appearing in the Matrix movies, the Shadowrun universe, Cyberpunk 2077 and more. In general, the concept is used to explore the effects of increasing digitization on society, human identity, and the implications of AI existing entirely within a virtual world.</p>
To start, we first need to install Stability Matrix, which can be downloaded from its website</a> or from its GitHub releases</a> page. The installation process is pretty straightforward and I highly recommend opting for a portable install. This keeps all data, such as models, within a single directory, making it much easier to transfer everything to a different drive or computer in the future.</p>
Once the application is installed, the next step is to set up a UI for generating images from prompts. There are multiple front-ends available from the packages menu. For this article, we will use Stable Diffusion WebUI Forge, as it is easy to use and more than sufficient for our very basic image generation use case.</p>

    
    
    
        
        The packages for the front-end listed that could be installed from Stability Matrix.</h4>
        
        
    </figcaption>
</figure>
We also need a model to generate images. For this we will use the Model Browser to find the right model for our use case. After searching for a model that is fine-tuned to generate cyberpunk imagery, I have settled on using the Splatter Punk - Neon Waves</code> model. Installing it is just a click away.</p>


    
    
    
        
        The 'Splatter Punk - Neon Waves' model used to generate the images.</h4>
        
        
    </figcaption>
</figure>
Now, with both a model and a front-end installed, we can start generating the images that we need. We want to generate pictures for four characters in a Cyberpunk themed world, based on four archetypes common in stories: the Hero, the Friend of the Hero, a Minion/Underling, and the Antagonist. While we could manually craft and fine tune the prompts, ChatGPT does a decent job of generating them for these archetypes:</p>

    
    
    
        
        Jax `Glitch` Varen (The Hero)</h4>
        
        
    </figcaption>
</figure>
"A brooding, determined man standing in a neon-lit alley of a futuristic city, wearing a sleek black jacket with subtle glowing circuitry patterns. His eyes reflect a mix of sadness and determination, and a faint holographic interface hovers around his wrist. The background features towering skyscrapers with digital ads, and rain softly blurs the neon glow. The atmosphere should feel tense and introspective, symbolizing betrayal and a relentless quest for justice."</p>

    
    
    
        
        Nova `Patch` Rhee (The Friend)</h4>
        
        
    </figcaption>
</figure>
"A cheerful and energetic woman in her late twenties, standing in a cluttered workshop filled with glowing wires, cybernetic components, and floating holograms. She wears mismatched tech goggles on her forehead, with brightly colored clothes accented by glowing accessories. A small, hovering drone buzzes beside her, reflecting her inventive and playful nature. The setting should be cozy yet chaotic, highlighting her technical genius and hidden vulnerabilities."</p>

    
    
    
        
        Cipher (The Minion/Underling)</h4>
        
        
    </figcaption>
</figure>
"A shadowy, humanoid figure made of fragmented code and shifting digital glitches, with piercing, glowing eyes. The body appears semi-transparent, constantly morphing between solid and pixelated forms. The background is a cold, digital void filled with streams of raw data flowing like rivers of light. The mood should be ominous and relentless, emphasizing Cipher’s role as an adaptive enforcer bound by hidden desires for autonomy."</p>

    
    
    
        
        Dr. Elara Voss (The Antagonist)</h4>
        
        
    </figcaption>
</figure>
"A powerful, composed woman standing at the center of a vast control room filled with holographic screens and pulsing data streams. She wears a high-collar, futuristic coat with minimalist design elements, her silver hair tied back in a severe style. Her expression is cold and calculating, with a subtle smirk of superiority. Behind her, the Matrix’s digital skyline stretches endlessly, symbolizing her control over the virtual world and her ambition for perfection."</p>
The images above were generated using the default parameters with the listed prompts. In fact, the portraits of Jax and Nova were created on the first attempt, while the others were selected from multiple generated images. For this particular model, the positive prompts were mostly sufficient, however adding a negative prompt, such as one to filter out NSFW content, can be useful.</p>
It has been incredible to see how much better the tooling and the models have become since these locally installable image generation models first were released. In the early days setups usually involved installing and managing many dependencies from the commandline, which is no longer required. Whether for storytelling, concept art, or creative exploration, AI-generated imagery even from a desktop can be just one install away.</p>


Livebook of Oa
2024-10-22T00:00:00+00:00
Code notebooks, which can provide an interactive combination of code, data, AI, text and visualizations, have seen increasing use in recent years. Livebook</a> is such a notebook environment that has some interesting features to explore. In this article I will write a basic Livebook notebook to show off some these features. This notebook will describe information from the Green Lantern series of comic books. In the Green Lantern series there exists a book called the Book of Oa</code> that contains knowledge about Green Lanterns and it can convey this in a variety of forms. Inspired by this, the notebook in this article will show information about Green Lanterns in a variety of ways: text, code, visualizations and even with some AI applications. </p>

    
    
    
        
        The Book of Oa from the Green Lantern series of comic books. Copyright DC Comics.</h4>
        
        
    </figcaption>
</figure>
If one wants to follow along with the full source of the notebook presented here it can be found in this repo on GitHub</a>. </p>
In order to run Livebook one needs to install</a> the Livebook app, either locally or in the cloud. The application is itself a web application that the user, or even multiple users, can connect to. Within the app one can create a new notebook, or open an existing one. </p>
In a new notebook the first step is usually installing some dependencies for the specific visualizations and AI tasks that we aim to do. Many of these can be added 'on demand' when we want to create them during the development, but here we just load in all the dependencies we need for the example notebook in one go:</p>
Mix</span>.install([
</span>  {</span>:kino_maplibre</span>, "</span>~> 0.1.12</span>"},
</span>  {</span>:kino_vega_lite</span>, "</span>~> 0.1.11</span>"},
</span>  {</span>:kino_bumblebee</span>, "</span>~> 0.5.0</span>"},
</span>  {</span>:exla</span>, "</span>~> 0.7.1</span>"}
</span>],
</span>  </span>config: </span>[
</span>    </span>nx: </span>[
</span>      </span>default_backend: </span>EXLA</span>.</span>Backend</span>,
</span>      </span>default_defn_options: </span>[</span>compiler: </span>EXLA</span>]
</span>    ]
</span>  ],
</span>  </span>system_env: </span>[
</span>    </span>XLA_TARGET: </span>"</span>cpu</span>",
</span>  ]
</span>
</span>)
</span></code></pre>
In Livebook the main language for code blocks is Elixir</a>, a functional programming language in which Livebook itself was written. These code blocks can be evaluated, which in this case results in  the atom:ok</code> being returned, denoting that this setup went well. Atoms in Elixir are constants whose value is their name. They are often used to note the state of an operation, in this case :ok</code> showing success, or distinct values, an example for which we will see later.</p>
Aside from code blocks, another type of element we can add to these notebooks is text. For example a brief part of the introduction in the notebook contains the following text using Markdown:</p>

"Green Lanterns are superheroes that are part of an intergalactic law enforcement agency called the Green Lantern Corps. They derive their powers through their Power Rings, which they aim to control through their willpower. The Power Rings give the Green Lanters various powers such as flight, creating forcefields and contructs made for energy.
The headquarters of the Green Lanterns are on the planet Oa, which is the home planet of the Guardians of the Universe. On the planet there is the Main Power Battery that powers all rings, as well as the Book of Oa (from which the name of this article is derived) which contains the laws and history of the Green Lantern Corps.
Aside from Markdown, Livebooks can have many other types of content. For example we can create a diagram for the First Appearance of a number of Green Lanterns using the built in support for Mermaid.js."</p>
</blockquote>
As mentioned previously, there are many other options for different types of content. For example creating diagrams using Mermaid.js</a> can be a breeze. The following block creates a diagram of the timelines of the first appearances of some of the Green Lanterns:</p>
timeline
</span>    title First Appearance
</span>    1959: Hal Jordan
</span>    1968: Guy Gardner
</span>    1972: John Stewart
</span>    1994: Kyle Rayner
</span>    2012: Simon Baz
</span>    2014: Jessica Cruz
</span>    2020: Sojurner Mullein
</span></code></pre>

    
    
    
        
        A Mermaid.js timeline of the first appearance of Green Lanterns in the comic books that is created with the above code.</h4>
        
        
    </figcaption>
</figure>
Code blocks can do more than just set up our notebook. For example we can define a struct to describe data relating to a Lantern. </p>
defmodule </span>Lantern </span>do
</span>  </span>defstruct </span>name: </span>"", </span>color: </span>:green</span>, </span>sector: </span>"", </span>home_town: </span>"", </span>appearances: 0
</span>end
</span></code></pre>
Without diving deep into Elixir semantics, the Lantern struct holds data about the name, color, sector, home town and appearances of a Lantern, with some basic default values. Notably the default value for the color of the Lanterns is green, denoted by the atom :green</code>.</p>
With these structs we can specify the data of six Green Lanterns from earth, and put them in a list named green_lanterns_from_earth</code>.</p>
hal_jordan =  %</span>Lantern</span>{</span>name: </span>"</span>Hal Jordan</span>", </span>sector: 2814</span>, </span>home_town: </span>"</span>Coast City, California</span>", </span>appearances: 5396  </span>}
</span>guy_gardner =  %</span>Lantern</span>{</span>name: </span>"</span>Guy Gardner</span>", </span>sector: 2814</span>, </span>home_town: </span>"</span>Baltimore</span>", </span>appearances: 1631 </span>}
</span>john_stewart=  %</span>Lantern</span>{</span>name: </span>"</span>John Stewart</span>", </span>sector: 2814</span>, </span>home_town: </span>"</span>Detroit, Michigan</span>", </span>appearances: 1866</span>}
</span>kyle_rayner =  %</span>Lantern</span>{</span>name: </span>"</span>Kyle Rayner</span>", </span>sector: 2814</span>, </span>home_town: </span>"</span>Los Angeles, California</span>", </span>appearances: 1696</span>}
</span>simon_baz =  %</span>Lantern</span>{</span>name: </span>"</span>Simon Baz</span>", </span>sector: 2814</span>, </span>home_town: </span>"</span>Dearborn, Michigan</span>", </span>appearances: 443</span>}
</span>jessica_cruz=  %</span>Lantern</span>{</span>name: </span>"</span>Jessica Cruz</span>", </span>sector: 2814</span>, </span>home_town: </span>"</span>Portland, Oregon</span>", </span>appearances: 431</span>}
</span>sojourner_mullein =  %</span>Lantern</span>{</span>name: </span>"</span>Sojourner Mullein</span>", </span>sector: 2814</span>, </span>home_town: </span>"</span>New York City, New York</span>", </span>appearances: 72</span>}
</span>green_lanterns_from_earth = [hal_jordan, guy_gardner, john_stewart, kyle_rayner, simon_baz, jessica_cruz, sojourner_mullein]
</span></code></pre>
As these Green Lanterns all have a hometown on Earth we can also specify the coordinates of these locations.</p>
hometown_coordinates = %{
</span>  "</span>coordinates</span>" => ["</span>37.865894, -122.498055</span>", "</span>39.299236, -76.609383</span>", "</span>42.331429, -83.045753</span>", "</span>34.052235, -118.243683</span>", "</span>42.322262, -83.176315</span>", "</span>45.523064, -122.676483</span>", "</span>40.730610, -73.935242</span>"],
</span>  "</span>name</span>" => ["</span>Coast City</span>", "</span>Baltimore</span>", "</span>Detroit</span>", "</span>Los Angeles</span>", "</span>Dearborn</span>", "</span>Portland</span>", "</span>New York City</span>"]
</span>}
</span></code></pre>
The above coordinates can be used to create a visualization of a map marked with the hometowns of the Lanterns. Although this map could be created programmatically, Livebook has the notion of smart cells</code> with which UI components can be rapidly created, without reaching for code. The following map was created using this feature.</p>

    
    
    
        
        A map of the hometowns of the Green Lanterns.</h4>
        
        
    </figcaption>
</figure>
In addition to Green Lanterns there are also Lanterns of other colors in the comics as well. Thankfully our Lantern</code> struct also allows us to specify these:</p>
atrocitus = %</span>Lantern</span>{</span>name: </span>"</span>Atrocitus</span>", </span>color: </span>:red</span>, </span>sector: 666</span>, </span>appearances: 321</span>}
</span>bleez = %</span>Lantern</span>{</span>name: </span>"</span>Bleez</span>", </span>color: </span>:red</span>, </span>sector: 33</span>, </span>appearances: 198</span>}
</span>red_lanterns = [atrocitus, bleez]
</span></code></pre>
We can combine the two lists of Lanterns into one with some Elixir code.</p>
lanterns = green_lanterns_from_earth ++ red_lanterns
</span></code></pre>
Now that we have all these lanterns defined, we can create a visualization that lists all lanterns with their respective appearances. The first step towards this to create a data structure combining together the lanterns names, number of appearances and color. The follow code creates this:</p>
lantern_appearances = </span>Enum</span>.reduce(lanterns, %{</span>lanterns: </span>[], </span>appearances: </span>[], </span>color: </span>[]}, </span>fn</span> lantern, acc ->
</span>  %{
</span>    </span>lanterns:</span> acc.lanterns ++ [lantern.name],
</span>    </span>appearances:</span> acc.appearances ++ [lantern.appearances],
</span>    </span>color:</span> acc.color ++ [lantern.color]
</span>  }
</span>end</span>)
</span></code></pre>
Which results in the following data when evaluated:</p>
%{
</span>  </span>color: </span>[</span>:green</span>, </span>:green</span>, </span>:green</span>, </span>:green</span>, </span>:green</span>, </span>:green</span>, </span>:green</span>, </span>:red</span>, </span>:red</span>],
</span>  </span>appearances: </span>[</span>5396</span>, </span>1631</span>, </span>1866</span>, </span>1696</span>, </span>443</span>, </span>431</span>, </span>72</span>, </span>321</span>, </span>198</span>],
</span>  </span>lanterns: </span>["</span>Hal Jordan</span>", "</span>Guy Gardner</span>", "</span>John Stewart</span>", "</span>Kyle Rayner</span>", "</span>Simon Baz</span>",
</span>   "</span>Jessica Cruz</span>", "</span>Sojourner Mullein</span>", "</span>Atrocitus</span>", "</span>Bleez</span>"]
</span>}
</span></code></pre>
This data can be used to create our visualization. Instead of using a smart cell, here we programmatically create the chart with the following code:</p>
VegaLite</span>.new(</span>title: </span>"</span>Lantern Appearances</span>")
</span>|> </span>VegaLite</span>.data_from_values(lantern_appearances, </span>only: </span>["</span>lanterns</span>", "</span>appearances</span>", "</span>color</span>"])
</span>|> </span>VegaLite</span>.mark(</span>:bar</span>)
</span>|> </span>VegaLite</span>.encode_field(</span>:x</span>, "</span>lanterns</span>", </span>type: </span>:nominal</span>)
</span>|> </span>VegaLite</span>.encode_field(</span>:y</span>, "</span>appearances</span>", </span>type: </span>:quantitative</span>)
</span>|> </span>VegaLite</span>.encode_field(</span>:color</span>, "</span>color</span>",
</span>  </span>type: </span>:nominal</span>,
</span>  </span>title: </span>"</span>Lantern Color</span>",
</span>  </span>scale: </span>[
</span>    </span>domain: </span>["</span>green</span>", "</span>red</span>"],
</span>    </span>range: </span>["</span>green</span>", "</span>red</span>"]
</span>  ])
</span></code></pre>

Evaluation of the above the code results in the following bar chart:</p>

    
    
    
        
        A bar chart of the number of lantern appearances in comic books.</h4>
        
        
    </figcaption>
</figure>
Finally, we can also add some AI functionality to round out our notebook. For example using a smart cell we can add a question answering system with only a few clicks:</p>

    
    
    
        
        A question answering system that can answer who Green Lanterns are based on our textual description.</h4>
        
        
    </figcaption>
</figure>
Even AI based image generation can be embedded with smart cells. We can use this to generate an image of a Green Lantern power ring.</p>

    
    
    
        
        An image of a Green Lantern power ring that has been generated.</h4>
        
        
    </figcaption>
</figure>
 
Of course this is just a very short glimpse of the things that a Livebook based notebook can do. One feature that is very useful in practice is that Livebook notebooks are saved as livemd</code> files which are a subset of markdown files. This makes them very suitable choice for writing documentation (such as for Github which renders livemd</code> files are markdown) and are easily used with version control. Another feature we did not touch on here is using real-time collaboration on the notebooks when using Livebook.</p>
Hopefully with this article I helped to shine some light on the nice features of the Livebook environment. Feel free to take a look on Livebook's website</a> for more details on using it.  </p>


Fellowship of Prolog
2024-06-25T00:00:00+00:00
Logic programming</a> with Prolog</a> can be a very effective way to find solutions to a number of problems. Within Prolog one can define a program by describing it with facts (that represent what we know), rules (that show how to derive new knowledge) and queries (that describe what we would like to know). By using queries on a knowledge base of facts and rules we can solve a variety of problems in many domains.</p>
This approach is quite powerful, but unfortunately Prolog as a programming language is often overlooked. One of the reasons for this is that it can be quite difficult for people without (logic) programming knowledge to break down problems into such terms. Even with experience in Prolog this process can be quite time consuming.</p>
Large Language Models (LLMs)</a>, such as Chat-GPT</a>, can provide an interesting way to tackle this issue. In this article I will aim to show how Chat-GPT (or another LLM) could help create and use Prolog programs to solve problems. For this I will be using "Lord of the Rings" themed logic puzzles.</p>

    
    
    
        
        Characters from the Fellowship of the Ring walking in a single line. This is the basic setup for the logic puzzles used in this article.</h4>
        
        
    </figcaption>
</figure>
Let's say we want to describe a situation in which the members of the Fellowship of the Ring are walking down a narrow path. The path is so narrow that no one can walk side by side. Each Fellowship member can only walk in a single line, one after another. The puzzles are based on the idea that given a number of conditions we would like to know in which order the members of the Fellowship are walking.</p>
First let's start off with a small number of conditions to show off the approach. We assume that only three members of the Fellowship are walking: Legolas, Gimli and Frodo. Our restrictions are that Legolas has to walk before Gimli and Gimli must be next to Frodo (this is the reverse order of what it can be seen in the above figure).</p>


In order to model this puzzle with Prolog, first we need to define the notion of the "single line order" that the Fellowship is walking in. In addition we need to define what terms such as before</code>, ahead</code>, etc mean within the context of this puzzle in Prolog. For one well versed in Prolog this can be a pretty routine task but in this case let's assume little to no Prolog knowledge for the person that aims to solve this puzzle. Instead we can ask Chat-GPT to come up with a translation of this problem to Prolog code. </p>
The Prolog code that we use in this article could likely be run with any number of Prolog implementations, such as SWI Prolog</a>. For this article we generate code to use with Tau prolog</a>. The benefit of using Tau prolog is that it can run in the browser and it has an online sandbox</a> available where code can be run without any installation. If one wants to follow along without any setup I highly recommend using this sandbox.</p>

To create the Prolog code for the simplified version of the puzzle we ask Chat-GPT the following question: </p>
"Can you write an example with tau prolog of a logic puzzle in which 3 members of the fellowship need to walk in a single line. Legolas has to be before Gimli and Gimli must be next to Frodo?"</p>
to which Chat-GPT gives us the code: </p>
% Load necessary library
</span>:- </span>use_module</span>(</span>library</span>(lists)).
</span>
</span>% Define the predicate to find the order
</span>fellowship_order</span>(Order) :-
</span>    </span>Order </span>= [_, _, _],  % There are three positions to fill.
</span>    </span>member</span>(legolas, Order),
</span>    </span>member</span>(gimli, Order),
</span>    </span>member</span>(frodo, Order),
</span>    </span>before</span>(legolas, gimli, Order),
</span>    </span>next_to</span>(gimli, frodo, Order).
</span>
</span>% Helper predicate to check </span>if </span>X is before Y in the List
</span>before</span>(X, Y, List) :-
</span>    </span>nth0</span>(IndexX, List, X),
</span>    </span>nth0</span>(IndexY, List, Y),
</span>    </span>IndexX </span>< IndexY.
</span>
</span>% Helper predicate to check </span>if </span>X is next to Y in the List
</span>next_to</span>(X, Y, List) :-
</span>    (   </span>nth0</span>(IndexX, List, X),
</span>        </span>nth0</span>(IndexY, List, Y),
</span>        </span>abs</span>(IndexX - IndexY) =:= </span>1
</span>    ).
</span>
</span>% Example query to find the order
</span>% ?- </span>fellowship_order</span>(Order).
</span></code></pre>

From this small example we can see the general outline of how such a logic puzzle can be solved with Prolog. Although we are not going to describe in detail the full syntax and semantics a Prolog program, as there are some excellent introductions</a> on this subject, we are going to briefly go over some concepts.</p>
A Prolog program is a knowledge base in which predicates, such as facts and rules, describe the domain. In our case the knowledge base defines this puzzle based on Lord of the Rings with three predicates: before</code>, next_to</code> and  fellowship_order</code>. The before</code> and next_to</code> predicates express whether for two members in a list are before or next to each others respectively. The X</code>, Y</code> and List</code> arguments expect variables (variable names are capitalized). The fellowship_order</code> uses these predicates to express what is a valid, according to our logic puzzle, order of the members of the Fellowship. The members, such as legolas</code> are atoms (they start with a lower case) and represent these characters in our domain. </p>
The query, fellowship_order(Order).</code> which was helpfully given to us by Chat-GPT, allows us to gather the valid orders according to the puzzle when querying the above knowledge base. </p>
We can test this easily by entering the above Prolog code in the Tau Prolog sandbox. After pasting in the program, we can press consult program</code>, which loads the knowledge base. As the goal that we want to query, we can enter fellowship_order(Order).</code>. Doing this will retrieve Order = [legolas,gimli,frodo]</code>, which is a valid answer to the logic puzzle.</p>

    
    
    
        
        Consulting the short Prolog example of the logic puzzle.</h4>
        
        
    </figcaption>
</figure>
Note that Order = [legolas,frodo,gimli]</code> is a valid answer as well. If we would ask the query again in the Tau Prolog sandbox, we will get this answer instead. If we query for a third (or any subsequent) time, we get a false</code>, which is Prolog's way of denoting that no answer could be found. This is due to Tau Prolog aiming to list all possible answers one by one given our query in this sandbox.</p>
Now that we have gone over a small example, let's see if this approach can work with a larger puzzle. While we can ask for the full Prolog code for a larger puzzle, in the version of Chat-GPT used at the time of writing, it had some trouble giving us a full and complete working program with lots of restrictions. Instead we can take the small example and add new constraints to it.</p>
Using this method we can create the following puzzle:</p>


% Load necessary library
</span>:- </span>use_module</span>(</span>library</span>(lists)).
</span>
</span>fellowship_order</span>(Order) :-
</span>    </span>Order </span>= [ _, _, _, _, _, _, _], 
</span>  	</span>member</span>(aragorn, Order),
</span>    </span>member</span>(frodo, Order),
</span>    </span>member</span>(sam, Order),
</span>    </span>member</span>(merry, Order),
</span>    </span>member</span>(pippin, Order),
</span>    </span>member</span>(gimli, Order),
</span>    </span>member</span>(legolas, Order),
</span>
</span>    % Defining relations based </span>on</span> problem statement
</span>    </span>before</span>(aragorn, pippin, Order), % Aragorn walks before Pippin.
</span>    </span>hobbit_neighbor</span>(frodo, Order),  % Frodo walks next to at least another Hobbit.
</span>    </span>hobbit_neighbor</span>(sam, Order),  % Sam walks next to at least another Hobbit.
</span>    </span>before</span>(gimli, legolas, Order),  % Gimli walks ahead of Legolas.
</span>    </span>next_to</span>(merry, pippin, Order), % Merry walks next to Pippin.
</span>    </span>after</span>(frodo, aragorn, Order). % Frodo walks after Aragorn.
</span>
</span>% Check </span>if </span>X is before Y
</span>before</span>(X, Y, [X|T]) :-
</span>    </span>member</span>(Y, T).
</span>before</span>(X, Y, [_|T]) :-
</span>    </span>before</span>(X, Y, T).
</span>
</span>% Check </span>if </span>X is after Y
</span>after</span>(X, Y, List) :-
</span>    </span>before</span>(Y, X, List).
</span>    
</span>% Ensure Sam is next to another hobbit
</span>hobbit_neighbor</span>(X, List) :-
</span>    </span>next_to</span>(X, frodo, List);
</span>    </span>next_to</span>(X, sam, List);
</span>    </span>next_to</span>(X, merry, List);
</span>    </span>next_to</span>(X, pippin, List).
</span>    
</span>% Check </span>if </span>X is next to Y
</span>next_to</span>(X, Y, [X, Y|_]).
</span>next_to</span>(X, Y, [Y, X|_]).
</span>next_to</span>(X, Y, [_|T]) :-
</span>    </span>next_to</span>(X, Y, T).
</span>    
</span>% Example query to find the order
</span>% ?- </span>fellowship_order</span>(Order).
</span></code></pre>
This puzzle has a number of new types of restrictions and more members of the Fellowship represent, but works the same way as the previous one. If we run it in the sandbox we can get an answer such as Order = [aragorn,frodo,sam,merry,pippin,gimli,legolas]</code> being returned. With an LLM we can even reverse the puzzle creation and get a textual description of the restrictions in the puzzle, as well as the puzzle itself. The following is the answer that ChatGPT returns when asked to describe the above puzzle in words:</p>


The Prolog puzzle presented involves arranging members of the Fellowship from "The Lord of the Rings" in a specific order based on given constraints. Here’s a step-by-step description:</p>
Objective</h4>
You need to determine the walking order of seven characters: Aragorn, Frodo, Sam, Merry, Pippin, Gimli, and Legolas.</p>
Given Constraints</h4>
Aragorn walks before Pippin.
</span>    Aragorn must come at some position earlier than Pippin.
</span>
</span>Frodo walks next to at least one other Hobbit.
</span>    Frodo must be adjacent to either Sam, Merry, or Pippin.
</span>
</span>Sam walks next to at least one other Hobbit.
</span>    Sam must also be adjacent to either Frodo, Merry, or Pippin.
</span>
</span>Gimli walks ahead of Legolas.
</span>    Gimli must come at some position earlier than Legolas.
</span>
</span>Merry walks next to Pippin.
</span>    Merry and Pippin must be directly adjacent in either order.
</span>
</span>Frodo walks after Aragorn.
</span>    Frodo must come at some position later than Aragorn.
</span></code></pre>
Key Definitions</h4>
Order:
</span>    A list of seven elements representing the walking order of the Fellowship members.
</span>    Example: [aragorn, frodo, sam, merry, pippin, gimli, legolas].
</span>
</span>before/3:
</span>    A predicate that checks if one character (X) appears before another (Y) in a list (Order).
</span>
</span>after/3:
</span>    A predicate that checks if one character (X) appears after another (Y) in a list (Order). It uses the before/3 predicate to establish this relation.
</span>
</span>hobbit_neighbor/2:
</span>    A predicate that checks if a character is next to at least one Hobbit in a list (Order). Hobbits include Frodo, Sam, Merry, and Pippin.
</span>
</span>next_to/3:
</span>    A predicate that checks if two characters are adjacent in a list (Order).
</span></code></pre>
Solution Mechanism</h4>
The fellowship_order/1 predicate defines the order and enforces all the constraints using the helper predicates.
</span>The list Order is constrained to have exactly the seven members of the Fellowship.
</span>The member/2 predicate ensures that all characters are included in the Order.
</span>The constraints (before/3, after/3, hobbit_neighbor/2, next_to/3) enforce the relative positions and adjacency as specified.
</span></code></pre>
Query</h4>
To find the correct order, you can query:</p>

</span>?- </span>fellowship_order</span>(Order).
</span></code></pre>
This will produce an Order that satisfies all the given constraints.</p>
Summary</h4>
This puzzle requires using logical relations to determine the correct sequence of characters while adhering to the given positional constraints. The use of predicates like before/3, after/3, hobbit_neighbor/2, and next_to/3 helps in logically deducing the valid walking order of the characters.</p>
</blockquote>
</div>
Such an approach can give us a very interactive way to evolve our problem description as well as the solution. We can give a text description of new constraints to the LLM, play around with the generated code in the sandbox, and go back to the LLM to refine the descriptions of the problem until we have the right problem statement and the solution. </p>
In this case the back and forth between the Prolog code and the text can help us further refine the puzzle. A helpful hint if one wants to experiment further: increasing the limit parameter that controls a maximum number of inferences can help to run more complex puzzles in the sandbox. Feel free to try it out with the sandbox!</p>
I hope you enjoyed this journey into Prolog from LLMs and back again and will consider using Prolog in such a way during your quest to solve other problems as well!</p>


AI at Home
2023-12-16T00:00:00+00:00
Large Language Models (LLMs), such as those that enable Chat-GPT, have been shown to be incredibly capable for language understanding and generation tasks. With the right prompt, they can answer questions, categorize input, rewrite pieces of text, perform sentiment analysis and more. </p>
As good as these tools are, many of them require data to be sent to a remote server and/or take some additional costs to run. This is due to the processing power required to use them and the proprietary nature of the models. However there exist models that can be run locally, even on relatively modest hardware, such as some of the LLaMA</a> models from Meta. Such locally runnable models can enable modern AI setups fully running at home, without the need for data being sent to another party. This article is a brief introduction on how to get one these models up and running.</p>

    
    
    
        
        A generated picture of an AI helping at home.</h4>
        
        
    </figcaption>
</figure>
The easiest solution that I have found for this purpose is the text-generation-webui</a> tool. As the name suggests it enables the use of language models by a web based UI and it can do so running only on the local machine. The setup is very straightforward: clone or download the repository, run the start script for your operating system, e.g.: start_wsl.bat</code>for running on WSL, and you are pretty much ready to go.</p>
This tool can use various models, but a nice one to get started is Llama-2-7B-GGUF model provided by TheBloke</a>, which is a 7 billion parameter LLaMa2 model. After loading it, we can simply start chatting.</p>

    
    
    
        
        A short conversation with the AI model.</h4>
        
        
    </figcaption>
</figure>
Although this model is on the smaller side it runs very adequately on a 2021 model Asus G14 laptop, and functions well for simple queries and conversations.</p>
There can be a lot of possibilities to explore with such a local setup, especially with larger, more capable models. It also provides an easy way for prototyping, as one can also use an API, that is a local drop-in replacement for the API of OpenAI, to interact with the model. </p>
I hope this article can help you get you started in exploring your AI use cases locally!</p>


Accidental Renaissance
2023-10-14T00:00:00+00:00
One interesting place to look for nice photographs is the Accidental Renaissance subreddit</a>. It is a forum where photographs that resemble Renaissance art, or other art movements that existed between the 14th and 19th centuries, are shared.</p>

    
    
    
        
        Photo of part of a kitchen in renaissance style, generated using mage.space.</h4>
        
        
    </figcaption>
</figure>
A related question is how to decide on what counts as Renaissance (or Baroque, Romanticist, etc) art for the photographs that would fall under the Accidental Renaissance subreddit. This is especially true for people with no in-depth knowledge on art movements. With the great results that deep learning based image classifiers can achieve and how easy it is using libraries such as fast.ai</a> to implement them, I thought it would be a good idea to create a solution for this question. In this article I aim to build a model that given a photo can decide whether a photograph could belong in the Accidental Renaissance subreddit.</p>
We are going to use two sources of data for this purpose. One is a collection of photos from the Accidental Renaissance subreddit, from which we can learn what makes for a good "Accidental Renaissance" photo. The other is a group of photos that are likely to be not accidental renaissance. For this we will use the following image data set from Kaggle</a>. This dataset contains images categorized into the following groups: Architecture, Arts and Culture, Food and Drinks and Travel and Adventure.</p>
With these two datasets we can start to create the classifier using fast.ai</a>. </p>
First part is importing the required dependencies.</p>
import </span>fastai
</span>from </span>fastai.vision.all </span>import </span>*
</span>from </span>random </span>import </span>sample
</span>from </span>ipywidgets </span>import </span>interact
</span>import </span>torch
</span></code></pre>
Next we define the paths for the images in the datasets.</p>
all_path = "</span>/mnt/c/data/accidental/</span>"
</span>renaissance_path = "</span>/mnt/c/data/accidental/renaissance</span>"
</span>architecture_path = "</span>/mnt/c/data/accidental/architecture</span>"
</span>art_and_culture_path = "</span>/mnt/c/data/accidental/artandculture</span>"
</span>food_and_drinks_path = "</span>/mnt/c/data/accidental/foodanddrinks</span>"
</span>travel_and_adventure_path = "</span>/mnt/c/data/accidental/travelandadventure</span>"
</span></code></pre>
While fast.ai has a function to gather all the image files from a directory, the version used for this experiment did not gather image files with the .webp extension. As many image files from Accidental Renaissance subreddit have a .webp extension, we will create a custom function to get all images files, including those of this type. </p>
def </span>get_all_image_files</span>(</span>path</span>, </span>recurse</span>=</span>True</span>, </span>folders</span>=</span>None</span>):
</span>    "</span>Get image files in `path` recursively, only in `folders`, if specified.</span>"
</span>    </span>return </span>get_files</span>(path, </span>extensions</span>=["</span>.webp</span>", "</span>.jpeg</span>", "</span>.gif</span>", "</span>.jpg</span>"], </span>recurse</span>=recurse, </span>folders</span>=folders)
</span></code></pre>
The files of each category can be retrieved using the above function and the relevant paths.</p>
renaissance_files = </span>get_all_image_files</span>(renaissance_path)
</span>architecture_files = </span>get_all_image_files</span>(architecture_path)
</span>art_and_culture_files = </span>get_all_image_files</span>(art_and_culture_path)
</span>food_and_drinks_files = </span>get_all_image_files</span>(food_and_drinks_path)
</span>travel_and_adventure_files = </span>get_all_image_files</span>(travel_and_adventure_path)
</span></code></pre>
Now that we have the list of files, it would be great to see how many of each type we have in the dataset.</p>
{"</span>Renaissance files</span>": </span>len</span>(renaissance_files), 
</span> "</span>Architecture files</span>": </span>len</span>(architecture_files),
</span> "</span>Arts and Culture files</span>": </span>len</span>(art_and_culture_files),
</span> "</span>Food and Drinks files</span>": </span>len</span>(food_and_drinks_files),
</span> "</span>Travel and Adventure files</span>": </span>len</span>(travel_and_adventure_files)
</span> }
</span></code></pre>
The above code will give use the following result: </p>
{'</span>Renaissance files</span>': </span>135</span>,
</span> '</span>Architecture files</span>': </span>8763</span>,
</span> '</span>Arts and Culture files</span>': </span>8531</span>,
</span> '</span>Food and Drinks files</span>': </span>7849</span>,
</span> '</span>Travel and Adventure files</span>': </span>8800</span>}
</span></code></pre>
It seems we have far more images in each category of the Kaggle dataset than those from the Accidental Renaissance subreddit. There are many possible ways to deal with an unbalanced dataset, but here we go for a simple sampling based solution. We create a sampled dataset from the Kaggle dataset that is as large as the image set of the subreddit, that we call our "regular" image set.</p>
sampled_regular_files = </span>sample</span>(architecture_files + art_and_culture_files + food_and_drinks_files + travel_and_adventure_files, </span>135</span>)
</span></code></pre>
Next we set up two directories, one for the renaissance files and one for the regular files respectively. We make sure to create the directories learning/renaissance</code> and learning/regular</code> if they do not yet exist. If they do exist already, the files within them will be deleted (which makes rerunning the experiment easier).</p>
def </span>delete_files</span>(</span>directory</span>):
</span>    </span>for </span>filename </span>in </span>os.</span>listdir</span>(directory):
</span>        file_path = os.path.</span>join</span>(directory, filename)
</span>        </span>if </span>os.path.</span>isfile</span>(file_path):
</span>            os.</span>remove</span>(file_path)
</span>
</span>learning_dir = "</span>learning</span>"
</span>learning_renaissance_dir = "</span>learning/renaissance</span>"
</span>learning_regular_dir= "</span>learning/regular</span>"
</span>if </span>not os.path.</span>exists</span>(learning_renaissance_dir):
</span>    os.</span>makedirs</span>(learning_renaissance_dir)
</span>else</span>:
</span>    </span>delete_files</span>(learning_renaissance_dir)
</span>if </span>not os.path.</span>exists</span>(learning_regular_dir):
</span>    os.</span>makedirs</span>(learning_regular_dir)
</span>else</span>:
</span>    </span>delete_files</span>(learning_regular_dir)
</span></code></pre>
With the directories in place we can copy all the files into their respective directories as the final part of our dataset setup.</p>
for </span>file </span>in </span>sampled_regular_files:
</span>    shutil.</span>copy</span>(file, learning_regular_dir)
</span>
</span>for </span>file </span>in </span>renaissance_files:
</span>    shutil.</span>copy</span>(file, learning_renaissance_dir)
</span></code></pre>
We can double check things to see if everything went well. First we check if the amount of files in the directories add up.</p>
files = </span>get_all_image_files</span>(learning_dir)
</span>len</span>(files)
</span></code></pre>
This should return 270. We can also check and remove any files that could not be read as image files using the following:</p>
failed = </span>verify_images</span>(files)
</span>failed.</span>map</span>(Path.unlink)
</span></code></pre>
Next we are going to set up a Datablock to get everything ready to run the learners using fast.ai. The categories will be labelled renaissance</code> and regular</code>.</p>
def </span>label_function</span>(</span>o</span>):
</span>    parent_name  = </span>Path</span>(o).parent.name
</span>    </span>if </span>parent_name == "</span>renaissance</span>":
</span>            </span>return </span>"</span>renaissance</span>"
</span>    </span>else</span>:
</span>            </span>return </span>"</span>regular</span>"
</span>
</span>data_block = </span>DataBlock</span>(
</span>    </span>blocks</span>=(ImageBlock, CategoryBlock), 
</span>    </span>get_items</span>=get_all_image_files, 
</span>    </span>splitter</span>=</span>RandomSplitter</span>(</span>valid_pct</span>=</span>0.2</span>, </span>seed</span>=</span>4</span>),
</span>    </span>get_y</span>=label_function,
</span>    </span>item_tfms</span>=[</span>RandomResizedCrop</span>(</span>128</span>, </span>min_scale</span>=</span>0.3</span>)]
</span>)
</span>
</span>dls = data_block.</span>dataloaders</span>(learning_dir)
</span></code></pre>
We can check our datablock setup by showing a batch of images and their labels from it.</p>
 dls.show_batch(max_n=8)
</span></code></pre>

    
    
    
        
        A batch of the datablock showing examples of the image dataset with their labels.</h4>
        
        
    </figcaption>
</figure>
The above code will return a batch such as this, if all went well.</p>
Now that we have the datablock set up we can do the learning. We are going to use the resnet34 model as a base with 5 iterations of fine tuning.</p>
learn = </span>vision_learner</span>(dls, resnet34, </span>metrics</span>=error_rate)
</span>learn.</span>fine_tune</span>(</span>5</span>)
</span></code></pre>
After 5 iterations, this particular model got a 0.486287 loss on the training set, a 0.595300 loss on the validation set and an error rate of 0.203704.</p>
So how does the model actually do in practice? One interesting way to evaluate it is to generate a few photos in a renaissance art style, with the nice AI image generation tools available. I have generated a few photos using mage.space</a> where I aimed for modern objects in a renaissance style, similarly to what might be photographed and posted in the Accidental Renaissance subreddit.</p>
For a generated photo of a kitchen in renaissance style the model got the right category predicted (i.e.: "renaissance"), with a probability of 0.7438. </p>

    
    
    
        
        A photo of a kitchen in renaissance style generated on mage.space.</h4>
        
        
    </figcaption>
</figure>
A generated photo of a car in a renaissance style got the right category predicted with a probability of 0.8643.</p>

    
    
    
        
        A photo of a car in renaissance style generated on mage.space.</h4>
        
        
    </figcaption>
</figure>
However interestingly enough the trained model has trouble recognizing actual renaissance (or baroque) paintings. The famous picture of the Last Supper</a> by Da Vinci got classified as "regular", just as Judith Beheading Holofernes</a> by Caravaggio. </p>
It would be interesting to hypothesize and investigate why this could be the case. Although they might be similar in art style, the paintings and the photos in the subreddit as well as the generated photos might differ in other aspects, such as the subject matter. The learning process is likely needing additional data, as 135 photos might be too low for a well fine tuned model. We could for example use exiting renaissance or baroque art to bolster the dataset. We could also investigate data augmentation to improve the training set.</p>


A Linked Data Based Advanced Credit Rationale
2023-06-23T00:00:00+00:00
Abstract</h1>
"A credit rationale forms an integral part of the credit granting and credit decision making process. Historically such credit rationales were written as a natural language document. This meant that automated handling of the meaning, i.e.: semantics, of such documents by machines was too complex to be done with sufficient accuracy. As a consequence, the creation and verification of a credit rationale is a time consuming and manual process. The solution that we propose is to create a Linked Data based credit rationale that we call the Advanced Credit Rationale (AdCR). Linked Data can ensure that the semantics in the credit rationale are both human and machine understandable. This enables the automated checking of the credit rationale for potential regulatory issues, as well as semantic based querying over a whole portfolio of such credit rationales. An issue that often hinders the use of a Linked Data based approach is that the creation of the knowledge graph can be challenging for domain experts. In this work we propose an approach that facilitates the creation of a credit rationale that is based on a Linked Data knowledge graph. We evaluate this approach with a prototype. We also demonstrate that with this approach the creation of a Linked Data based credit rationale should not take more time than a natural language based credit rationale. In addition we show that the structures that a Linked Data based credit rationale provides are generally seen as helpful."</p>
Authors</h1>
Newres Al Haider, Keng Ng, Ali Hashmi, Lauma Veidemane, Diederik Schut </p>
Presented at</h1>
1st Workshop of Knowledge Graphs for Semantics-driven Systems Engineering (KG4SDSE) 
A Workshop at CAiSE2023 
Appears in Advanced Information Systems Engineering Workshops. CAiSE 2023. Lecture Notes in Business Information Processing, vol 482</a></p>
Related Project</h1>
Legal Banking Compliance</a></p>


Dunes
2022-09-18T00:00:00+00:00
Stable Diffusion</a> is one of the latest models that is capable of translating a piece of text, such as "arid, desert dunes" into great looking images. Unlike some other tools and models, Stable Diffusion can be installed and run locally on a desktop machine (see this repo and instructions</a>).</p>
One of the great features of the Stable Diffusion model is the ability to combine a text input with a pre-existing image to generate new images. In this article I aim to dive into this feature by starting with a photo of dunes that I took a while back. I will use descriptions based on sci-fi worlds with dunes, notably Arrakis from Dune</a> and Tatooine from Star Wars</a>, to create images from the original photo that look like as if they were from these worlds.</p>

    
    
    
    
    
        
        
        A sequence of images (from left to right): the original picture of dunes in the Netherlands (Soester Duinen), a generated image of Arrakis and a generated image of Tatooine.
        
    </figcaption>
</figure>
The original photo itself has been taken in the sand dunes of Soester Duinen</a> in the Netherlands. The sand here has been deposited during the last ice age which came to lay bare due to intensive grazing during the middle ages. Almost all such areas in the Netherlands have been reclaimed from the sand, with the remaining dunes of the Soester Duinen now being maintained as geographical monuments.</p>

    
    
    
    
    
        
        
        The original photo I have taken of dunes in Soester Duinen.
        
    </figcaption>
</figure>
Given this image, with the help of Stable Diffusion, I had the goal to transform it to sand dunes that one could find in a sci-fi setting. </p>
First I tried to get an image that looks more like the planet Arrakis, also known as Dune</a>. One of the characteristics of this planet is that it is incredibly arid. This means that in order to get something that resembles Arrakis, with less vegetation and clouds, I need to provide a text prompt that emphasizes the harsh arid nature of Arrakis.</p>
I have used the prompt "The planet Arrakis also known as Dune, arid, desert dunes, desolate, cloudless blue sky" to transform the image to the one that follows:</p>

    
    
    
    
    
        
        
        An image of Arrakis, the planet in the novel Dune, that was generated.
        
    </figcaption>
</figure>
The Stable Diffusion model takes a number of parameters aside from the text prompt to generate images. One that is specific to using a starting image is called Img2Img strength, which defines how much weight should be given to the initial image when generating new images. Lower scores make the generated image look closer to the initial image, while higher scores let the model dream up new images more freely. For the images generated the value 0.75 was used (unless noted otherwise). One can quite well see the effect of the base image in the layout of the new one that is made to look like Arrakis e.g.: like how the sand dunes have replaced the vegetation in the background.</p>
It can generally take some trial and error to find the right parameters to generate images that fit the creators vision. Nonetheless it is quite amazing how many good looking images can be generated just by playing around with the parameter and prompt selection.</p>
While for Arrakis it took quite a long text prompt get close to the results that I wanted it is quite different when trying a Star wars related prompt. The prompt: "Tatooine, from Star Wars" is itself enough to help generate the following image from our original dune photo:</p>

    
    
    
    
    
        
        
        An image of Tatooine, a planet from Star Wars, that was generated.
        
    </figcaption>
</figure>
As one can see even from a short prompt, it is possible for the model to generate images that look like they were taken as photographs from the planet Tatooine from Star Wars. It seems the system has a notion of how the architecture of buildings look like on the planet in this setting and is capable of generating images of such buildings into the photograph.</p>
It is also possible to go beyond the well-known fictional planets and let the model dream up a setting.</p>
The prompt "Cyberpunk settlement, detailed" gave the following results:</p>

    
    
    
    
    
        
        
        A cyberpunk styled settlement that was generated from the original image. Generated with the img2img strength of 0.65.
        
    </figcaption>
</figure>
I hope I have given at least a small preview or what is possible with Stable Diffusion's img2img generation. It is quite a fun project to take photographs and modify them using text prompts to all kinds of imaginary settings. Images of the next sci-fi setting set in the dunes could be just one prompt and one photograph away.</p>


Star Wars Trek
2022-08-26T00:00:00+00:00
Recently I started to do the latest version of the course Practical Deep Learning for Coders</a> from fast.ai</a>. I am very much enjoying the hands-on approach of the course and it is quite amazing to see how a deep learning based image classifier could be built with very little code. In the first chapter of the book that accompanies the course</a> a model is trained to recognize whether an image depicts a bird or a forest. In this article, as an exercise, I will instead create a model that can recognize if an image of a spaceship is from Star Wars or from Star Trek.</p>

    
    
    
    
    
        
        
        A star destroyer from Star Wars on the left and the Enterprise from Star Trek on the right. Star Wars is the copyright of Disney and Star Wars is the copyright of Paramount Pictures.
        
    </figcaption>
</figure>
I will list all the code I used for creating and using this model in this article, with a brief description after each code fragment on what it does. The code is very similar to the code used in the the first chapter of the FastAI book, as in both cases we are aiming to recognize whether an image belongs to one of two categories, with the same setup. If one wants to follow along, I can highly recommend using a service such as Colab</a> to get started quickly but a local install also does work. Chapter 1 of the book</a> is directly available on Colab as well. </p>
Now let's get to the code used:</p>
! [ -e /content ] && pip install -Uqq fastbook
</span>import </span>fastbook
</span>fastbook.</span>setup_book</span>()
</span></code></pre>
The first part is installing and setting up all the dependencies. Assuming we are working in Colab we need the first line to install the dependencies. If we work in our local (virtual) environment we can just do a pip install fastbook</code> instead.</p>
from </span>fastbook </span>import </span>*
</span></code></pre>
The next part is importing all the things we will need from fastbook. For the purposes of this small tutorial we will just import everything.</p>
searches = '</span>star wars ship</span>','</span>star trek ship</span>'
</span>path = </span>Path</span>('</span>star_wars_or_trek</span>')
</span>
</span>if </span>not path.</span>exists</span>():
</span>    </span>for </span>o </span>in </span>searches:
</span>        dest = (path/o)
</span>        dest.</span>mkdir</span>(</span>exist_ok</span>=</span>True</span>, </span>parents</span>=</span>True</span>)
</span>        results = </span>search_images_ddg</span>(</span>f</span>'{o}</span> photo</span>')
</span>        </span>download_images</span>(dest, </span>urls</span>=results[:</span>200</span>])
</span>        </span>resize_images</span>(dest, </span>max_size</span>=</span>400</span>, </span>dest</span>=dest)
</span></code></pre>
Next we are going to gather the images based on which we will create and test our classifier. The above code will set up a directory called star_wars_or_trek</code>, assuming it does not exist yet, and will search for images using the phrase star wars ship</code> and star trek ship</code> using DuckDuckGo</a>. The found images will be downloaded in sub-directories called star wars ship</code> and star trek ship</code> containing the respective images. Finally we are going to resize the images that we download to a comparable maximum size. </p>
failed = </span>verify_images</span>(</span>get_image_files</span>(path))
</span>failed.</span>map</span>(Path.unlink)
</span></code></pre>
The next step is verifying that all the images we got are valid image files, as things can go wrong during search and download. If they are not valid images we can remove them from our dataset.</p>
dls = </span>DataBlock</span>(
</span>    </span>blocks</span>=(ImageBlock, CategoryBlock), 
</span>    </span>get_items</span>=get_image_files, 
</span>    </span>splitter</span>=</span>RandomSplitter</span>(</span>valid_pct</span>=</span>0.2</span>, </span>seed</span>=</span>42</span>),
</span>    </span>get_y</span>=parent_label,
</span>    </span>item_tfms</span>=[</span>Resize</span>(</span>192</span>, </span>method</span>='</span>squish</span>')]
</span>).</span>dataloaders</span>(path)
</span></code></pre>
The datablock is where all the elements are setup that are required for learning our model. It specifies that we want to learn from images and want to derive categories from it, i.e. whether an image is a Star Wars ship or a Star Trek ship. It uses the data from the files that we have downloaded. </p>
One very important aspect of creating a model is to ensure its predictions are accurate. A way we can test it is to set some portion of the data aside that we will use for evaluation as opposed to learning. In this case we use 20% of the data randomly selected for evaluation.</p>
We also specify that the label for each images can be derived from the directory that they are in. Finally we aim to apply a transform to the images, to standardize them in a way that helps the training of the model. </p>
dls.show_batch(max_n=6)
</span></code></pre>
A good way to check if our datablock is setup correctly is to show a batch images, in this case six, from our datablock. This can give us a set of images, such as the one below, that we can visually inspect before we start our learning.</p>

    
    
    
    
    
        
        
        A batch of six images from our dataset of ships that we have labelled either a Star Wars ship or a Star Trek ship.
        
    </figcaption>
</figure>
learn = </span>vision_learner</span>(dls, resnet18, </span>metrics</span>=error_rate)
</span>learn.</span>fine_tune</span>(</span>3</span>)
</span></code></pre>
The above two lines kick off the actual learning, i.e. the creation of a model that can differentiate between a ship from Star Wars and Star Trek, based on our setup of the datablock. One of the great things for image based models is that there are pre-trained models that exist, such as resnet18</code> that have been trained on a lot of images. This means that we do not have to start our image learning from scratch. Instead we can use this existing model as a starting point and fine tune it to our task at hand: the recognition of the right class of spaceship. </p>
Here we just do 3 iterations of fine tuning. The output from this fine tuning can be seen below. The results will vary for each run of fine tuning, but this will hopefully illustrate the process:</p>
epoch	train_loss	valid_loss	error_rate	time
</span>0	1.236515	0.963507	0.323944	</span>00:07
</span>epoch	train_loss	valid_loss	error_rate	time
</span>0	0.531795	0.751966	0.281690	</span>00:10
</span>1	0.419798	0.844729	0.225352	</span>00:10
</span>2	0.312620	0.631814	0.197183	</span>00:10
</span></code></pre>
In this case model was trained on desktop with a GPU but doing this on Colab is also very fast. We can get an error rate at around 0.2 with this setup which is good enough for our short article. That said it would be interesting exercise for the future to see how we could get this error rate down or to examine what are the examples where the model finds it difficult to predict the right category.</p>
Now that we learned our model we would like to put it to use by giving it an image to classify. We have two options on how to do this:</p>
uploader = widgets.</span>FileUpload</span>()
</span>uploader
</span></code></pre>
When using a notebook we can have a widget with a file selector, with which we can upload the image we would like to classify.</p>
uploader = </span>SimpleNamespace</span>(</span>data </span>= ['</span>images/stardestroyer.jpeg</span>'])
</span># uploader = SimpleNamespace(data = ['images/enterprise.webp'])
</span></code></pre>
We can otherwise just load in the image from our (local) drive as well. Here one line is commented out so we could quickly switch between two options for images. </p>
img = PILImage.</span>create</span>(uploader.data[</span>0</span>])
</span>is_star_wars,</span>_</span>,probs = learn.</span>predict</span>(img)
</span>print</span>(</span>f</span>"</span>What ship is this?: </span>{is_star_wars}</span>.</span>")
</span>print</span>(</span>f</span>"</span>Probability it's a star wars ship: </span>{probs[</span>1</span>].</span>item</span>()</span>:.6f</span>}")
</span>print</span>(</span>f</span>"</span>Probability it's a star trek ship: </span>{probs[</span>0</span>].</span>item</span>()</span>:.6f</span>}")
</span></code></pre>
The final part is taking the image that we now added and asking the learned model to predict what kind of ship it is. With the above code we will print out both the category of the ship as well as the probabilities attached to the category. This will give an indication of how confident the model is in the prediction.</p>
If we use the image of a Star Destroyer from Star Wars, that is displayed on the left at the start of this article, our model will predict with very high confidence that it is a ship from Star Wars.</p>
What ship is this?: star wars ship.
</span>Probability it's a star wars ship: 0.999536
</span>Probability it's a star trek ship: 0.000464
</span></code></pre>
Similarly, if we use the image of Enterprise from Star Trek, the model will have classify it correctly with very high probabilities. </p>
What ship is this?: star trek ship.
</span>Probability it's a star wars ship: 0.000007
</span>Probability it's a star trek ship: 0.999993
</span></code></pre>
In both cases the model can classify these iconic spaceships really well. It is really cool to see how little code is required to create and use a model for these type of predictions with fast.ai, which I think it is pretty amazing. I can not recommend the book/course Practical Deep Learning for Coders</a> enough and will definitely hope to dive deeper as I go along.</p>


Raku Parrot
2022-08-06T00:00:00+00:00
One of the programming languages that I have stumbled upon not too long ago is Raku</a>. This language, which was previously known as Perl 6, is chock-full of really cool features. It has support for programming in both object oriented and functional paradigms, optional gradual typing, a nice story for async programming and more. </p>
A really nice feature of Raku is that Grammars</a> are a core part of the language. Grammars allow for a great way to interpret textual information. When programming there are a lot of use cases where a specific set of patterns need to be recognized and used from text. An interpreter for a format such as JSON</a> is a good example. From the text version of a JSON file elements such as arrays, objects, numbers, strings, etc. needs to be recognized. In addition a representation of them needs to be built for manipulation and use within a programming language. </p>
In order to demonstrate this grammar feature of Raku, we are going to implement a small program to parrot phrases back to us. The grammar inside this program describes short three word (sometimes a bit irregular) sentences like "I like sushi." or "They eat pear.". Our program will parrot these phrases back to us twice, first just as text and the second time with certain words being replaced by emojis. </p>
To give a preview of what this code will do, if we call it from the command line given the input: </p>
I like sushi.
</span></code></pre>
we get back:</p>
I like sushi.
</span>I like 🍣.
</span></code></pre>
if we would use an input like: </p>
They eat pear.
</span></code></pre>
we get back:</p>
They eat pear.
</span>They eat 🍐.
</span></code></pre>
In cases where the our parroting program does not understand what we enter, for example for the sentence: </p>
This is too complex!
</span></code></pre>
we get the phrase back:</p>
Squaaawk can not understand!
</span></code></pre>
To show how this is implemented, first we will list the full code in Raku and we will go over its elements piece by piece. If one wants to follow along Replit</a> offers a way to see the Raku language in action without any installation. For a local installation the Rakudo</a> implementation or the Rakudo Star bundle</a> are great starting points. </p>
grammar ParrotHearing</span> {
</span>    </span>rule</span> TOP { <personalPronoun> <parrotVerb> <parrotNoun><ending>}
</span>    </span>token</span> personalPronoun { "</span>I</span>" | "</span>You</span>" | "</span>He</span>" | "</span>She</span>" | "</span>It</span>" | "</span>We</span>" | "</span>They</span>"| "</span>Me</span>" | "</span>Him</span>" | "</span>Her</span>" | "</span>Us</span>" | "</span>Them</span>" }
</span>    </span>token</span> parrotVerb { \w+ }
</span>    </span>token</span> parrotNoun { \w+ }
</span>    </span>token</span> ending { "</span>.</span>" | "</span>!</span>" | "</span>?</span>" }
</span>}
</span>
</span>sub </span>convertToEmoji</span>($</span>string</span>)
</span>{
</span>    </span>my </span>$</span>parsedValue</span> = uniparse($</span>string</span>.</span>uc</span>);
</span>    </span>if</span> ($</span>parsedValue</span>) {</span>return </span>$</span>parsedValue</span>}
</span>    </span>else</span> {</span>return </span>$</span>string</span>}
</span>}
</span>
</span>class </span>EmojiActions</span> {
</span>
</span>    </span>method </span>TOP</span> ($</span>/) {
</span>        my $personalPronoun = $<personalPronoun>;
</span>        my $parrotVerb= $<parrotVerb>.made;
</span>        my $parrotNoun= $<parrotNoun>.made;
</span>        my $ending= $<ending>;
</span>        make ( "$personalPronoun $parrotVerb $parrotNoun$ending")
</span>    }
</span>
</span>    method parrotVerb ($/</span>) {
</span>       make (convertToEmoji($</span>/))
</span>    }
</span>
</span>    method parrotNoun($/</span>) {
</span>       make (convertToEmoji($</span>/))
</span>    }
</span>}
</span>
</span>sub MAIN($name) {
</span>
</span>    my $parrotParsed =  ParrotHearing.parse($name.Str, actions => EmojiActions);
</span>    if ($parrotParsed) {say $parrotParsed.Str;
</span>                       say $parrotParsed.made.Str}
</span>    else {say "Squaaawk can not understand!"};
</span>}
</span></code></pre>
The first part of the implementation is defining the Grammar of the language that the parrot hears that we call ParrotHearing. Grammars in Raku are a way to organize Regular Expressions (Regexes)</a>. Without going into the formal definition of Regular Expressions</a> they can be used to define patterns of characters in text. These patterns can be utilized in various way, such as searching, replacing and transforming text. </p>
The basic elements of the grammar are regex objects. These contain the regex patterns that we use. In Raku these regular expressions are defined with their own domain specific language. For example one regex object our grammar will be using is { \w+ }</code>. The pattern in this object, \w+</code>,  matches a single word character, consisting of  letters, digits or underscores, one or more times. For example if we have the text "These are all w0rds_ "</code> it will match four times. </p>
Another regex object that we will use is { "I" | "You" | "He" | "She" | "It" | "We" | "They"| "Me" | "Him" | "Her" | "Us" | "Them" }</code> which contains a pattern that will match any of the listed personal pronouns that is capitalized. This means that it will match the text of "You", "They" and "Them", but not "i", "apple" or "them".</p>
Next let's take a look at how the ParrotHearing grammar is organizing the regular expressions. The sentences that that this grammar defines are very basic: they have the form of a personal pronoun, followed by verb, a noun and an ending. In fact, the verbs and nouns that the Parrot accepts are very simplified: they are the pattern \w+</code> that we described above that would match most words. The personal pronouns are those that the regex object { "I" | "You" | "He" | "She" | "It" | "We" | "They"| "Me" | "Him" | "Her" | "Us" | "Them" }</code> will match while the ending is one of the .</code>, !</code> and ?</code> characters.</p>
The final piece is a function that replaces the nouns and verbs in a sentence with the emoji equivalent if it can. If no replacement can be found it leaves the text unchanged. When parsing we can use this function as part of an action to perform. This allows us to tie everything together in our MAIN</code> function (also called a subroutine in Raku). This parses in the text from the command line, prints the text twice (can be done by the say</code> function in Raku). The first time it will print the text and the second time it will print the text with the emoji replacements. In the case when it can not parse the input it will print "Squaaawk can not understand!"</code> instead.</p>
To run everything, if the code is in a file called main.raku one can call raku main.raku "I love sushi."</code> or some other sentence to get started. Feel free to give it a try on Replit</a> to see what the code parrots back!</p>


Data Oriented Domain Design
2022-04-05T00:00:00+00:00
One of the most difficult parts of developing software for a particular task is understanding the (business) domain in which the software needs to operate. This is due to the fact that software engineers often lack the expertise in complex areas such as health, law, finance, etc in which the software they create needs to solve problems. A similar situation holds for domain experts. For example, while lawyers are by definition well versed in the legal domain, it is not necessarily the case that they are also software engineers. This lack of knowledge can make developing software quite difficult. To continue with our legal example, suppose we want to create an application that represents and searches laws. From the software engineers side, the legal terminology can be quite burdensome to understand and it might not be very clear to them what aspects of it would be important for them to implement. On the other hand, it is important to be able to make it understandable to legal experts, without any software engineering knowledge, what the software is doing in relation to the concepts in their domain.</p>
Various solutions have been proposed for this problem, such as Domain Driven Design</a> and formalizing the domain using Linked Data Ontologies</a>.
These methodologies can be quite complex, which they need to be in order to capture the nuances of the domain and descriptions of what is possible. This makes the important process of domain formalisation often quite daunting. In this article, I will show a very lightweight approach, that can be applied as a starting point towards formalising the domain. The core idea is to use lightweight data notation languages, such as JSON</a> and EDN</a> to represent elements of the domain. The elements of these languages, such as the notations for sequences, key and value pairs, etc would then be used as a common language for both the software engineer and domain expert to explain the problem and how the software is used to solve it.</p>
For the purpose of this article I will call this method Data Oriented Domain Design (DODD). Not coincidentally the subject that we aim to represent are a few elements of the Dodd-Frank Wall Street Reform and Consumer Protection Act</a>, which we will simply refer as the Dodd-Frank law in this context. </p>
In this article I will make use of the EDN</a>(Extensible Data Notation) data notation language to illustrate the ideas of this Data Oriented Domain Design (DODD). EDN is a language where information is represented through a set of elements as values. The elements are common to many programming languages. For example here are elements for representing text, time, numbers as well as collections, such as lists, sets, etc. of these elements. </p>
EDN is a subset of the programming language Clojure</a>. A large part of a Clojure program is manipulating information expressed in EDN. Due to this reason Clojure is often called a data-oriented or data-driven language. With Data Oriented Domain Design we are going to use this "data orientation paradigm" not just for organizing the software system but also for creating a common language in which software engineers and domain experts can communicate.</p>
In order to make things more concrete lets imagine a scenario where small application is needed to be built that aims to search for definitions within legal documents. One of the nice aspects of many legal texts is that they often have a section of definitions that the reader can refer to. Our application will aim to retrieve these definitions based on a given criteria. As an example use case, given the acronym DODD of our approach, we will use the text of the Dodd-Frank Wall Street Reform and Consumer Protection Act</a>, which we will simply refer as the Dodd-Frank law in this context. </p>
I am going to preface this by saying that "I am not a lawyer" and I am looking at this application from a software- and knowledge engineers perspective. I will simplify out a LOT of the intricacies of legal text search and representation. That said as mentioned this perspective should be illustrative of the issues when developing software for such a new domain and how Data Oriented Domain Design could be a good starting point.</p>
The text of this law has multiple sections for definitions, but here in particular we are looking at the first ten definitions outlined in Section 2 of the Dodd-Frank act. </p>
SEC. 2. <<NOTE: 12 USC 5301.>> DEFINITIONS.
</span>
</span>    As used in this Act, the following definitions shall apply, except 
</span>as the context otherwise requires or as otherwise specifically provided 
</span>in this Act:
</span>            (1) Affiliate.--The term ``affiliate'' has the same meaning 
</span>        as in section 3 of the Federal Deposit Insurance Act (12 U.S.C. 
</span>        1813).
</span>            (2) Appropriate federal banking agency.--On and after the 
</span>        transfer date, the term ``appropriate Federal banking agency'' 
</span>        has the same meaning as in section 3(q) of the Federal Deposit 
</span>        Insurance Act (12 U.S.C. 1813(q)), as amended by title III.
</span>
</span>[[Page 124 STAT. 1387]]
</span>
</span>            (3) Board of governors.--The term ``Board of Governors'' 
</span>        means the Board of Governors of the Federal Reserve System.
</span>            (4) Bureau.--The term ``Bureau'' means the Bureau of 
</span>        Consumer Financial Protection established under title X.
</span>            (5) Commission.--The term ``Commission'' means the 
</span>        Securities and Exchange Commission, except in the context of the 
</span>        Commodity Futures Trading Commission.
</span>            (6) Commodity futures terms.--The terms ``futures commission 
</span>        merchant'', ``swap'', ``swap dealer'', ``swap execution 
</span>        facility'', ``derivatives clearing organization'', ``board of 
</span>        trade'', ``commodity trading advisor'', ``commodity pool'', and 
</span>        ``commodity pool operator'' have the same meanings as given the 
</span>        terms in section 1a of the Commodity Exchange Act (7 U.S.C. 1 et 
</span>        seq.).
</span>            (7) Corporation.--The term ``Corporation'' means the Federal 
</span>        Deposit Insurance Corporation.
</span>            (8) Council.--The term ``Council'' means the Financial 
</span>        Stability Oversight Council established under title I.
</span>            (9) Credit union.--The term ``credit union'' means a Federal 
</span>        credit union, State credit union, or State-chartered credit 
</span>        union, as those terms are defined in section 101 of the Federal 
</span>        Credit Union Act (12 U.S.C. 1752).
</span>            (10) Federal banking agency.--The term--
</span>                    (A) ``Federal banking agency'' means, individually, 
</span>                the Board of Governors, the Office of the Comptroller of 
</span>                the Currency, and the Corporation; and
</span>                    (B) ``Federal banking agencies'' means all of the 
</span>                agencies referred to in subparagraph (A), collectively.
</span></code></pre>
Now lets consider the case of an application searching for definitions in legal text. One of the features we would like to do is that given the exact name of the term, such as "Board of Governors", "Bureau", etc, the text of its definition is found. The name of the term can be represented as data with a string of characters. In Clojure, and many other languages, this is denoted as the text between quotation marks (""</code>). For example, the input for our search can be given as:</p>
"</span>Board of Governors</span>"
</span></code></pre>
Next we have to examine on how to represent the output (the result) of our search. In most cases we would like to have a sequence of results that indicate the found definitions. This is due to a number of reasons. First, a specific term could be defined with multiple definitions over a variety of documents. Second, it provides us a straightforward way to represent the results in the case when there are no results are found. In this case we can return an indicator of a sequence of 0 elements. In Clojure such a sequence of elements can be indicated by elements in between square brackets []</code>. Other data interchange formats and languages call such ordered sequence by different terms: lists, arrays, etc. Such representations are very common in (programming languages) but for in this article we will stick with the EDN definitions. </p>
In our first version of our program we will just return the sequence of the found texts that describes the definition. To give a concrete example lets assume we search for the term "Board of Governors" in the above partial document. The input for our program is the string of characters indicating this term:</p>
"</span>Board of Governors</span>"
</span></code></pre>
and the output would be:</p>
[ "</span>(3) Board of governors.--The term ``Board of Governors'' means the Board of Governors of the Federal Reserve System.</span>"]
</span></code></pre>
To show an example where we would not find any definitions in the above text, if we search for the term "Central Bank" in the above fragment using the input</p>
"</span>Central Bank</span>"
</span></code></pre>
we would get an empty sequence as a result:</p>
[]
</span></code></pre>
Giving examples such as this should already give a good indication on how the code could be structured. Just as importantly, given some information on the basic notation of sequences and strings, legal domain experts could understand the program is aiming to achieve, just by looking at a few of such examples and verify whether we are on the right track.</p>
Of course the above example is a very minimal abstraction. Let's try to expand upon it, to capture a few more elements of the legal domain. </p>
The location in which the searched terms are found is also quite important. A program might search legal terms over multiple legal documents, and if found, people would likely want to know exact location in the source material to get more context. This means that we want to represent a few additional values in our search result. For example if the definition was found in a law, we would also want to have the title of the law, the section in which the definition was found, the URL of where the law could be read, etc.</p>
In EDN the format to represent such key-value pairs is called map, represented by pairs of elements between curly brackets: {}</code>. For example if we want to express the term description with a key-value pair, with both the key and value being a string, we could write:</p>
[{ "</span>description</span>" "</span>(3) Board of governors.--The term ``Board of Governors'' means the Board of Governors of the Federal Reserve System.</span>"}]
</span></code></pre>
In other languages such maps are called as object, record, struct, dictionary, hash table, etc but the general concept is the same. </p>
The main question is of-course what additional elements we want add here and what we would like to name the keys of these elements. To reiterate, "I am not a Lawyer", but we can look towards the ways of how laws are cited to figure out what additional information would make sense from a legal perspective. Thankfully there are some descriptions online on how to cite laws, such on the site of the University of Cincinnati</a> and the Cornell University Law</a>.</p>
Given this information, we will add the terms "title of the act", "public law number" "statute", "year of enaction" and "url" of the law where this definition was found to our search result. Below is the expanded example:</p>
[{ "</span>description</span>" "</span>(3) Board of governors.--The term ``Board of Governors'' means the Board of Governors of the Federal Reserve System.</span>"
</span>  "</span>title-of-act</span>" "</span>H.R.4173 - Dodd-Frank Wall Street Reform and Consumer Protection Act</span>"
</span>  "</span>public-law-number</span>" "</span>111-203</span>" 
</span>  "</span>statute</span>" "</span>1387</span>"
</span>  "</span>year-of-enaction</span>" "</span>2010</span>"
</span>  "</span>url</span>" "</span>https://www.congress.gov/bill/111th-congress/house-bill/4173/text?r=1</span>"}]
</span></code></pre>
Now this might be sufficient, but again there are some features we could use to make this a better representation. First, we use a lot of strings of characters when we ideally want to specify a number or numbers. In EDN, as well as in JSON and other formats, you generally have a bit more precision and safety by expressing these elements as numbers (or more precisely integers). By the program that we intend to build, this helps us automatically invalidate certain wrong values and helps us better describe the intent of what is allowed. For example if we write the string "start"</code> instead of a number such as '111' for the law number, this program should handle this by not allowing such scenaros to occur that contradict the rules of the domain. The extra precision allows us to more precisely declare the range of laws, with a number denoting the starting and the ending point with the keys "from"</code> and "to"</code>. </p>
[{ "</span>description</span>" "</span>(3) Board of governors.--The term ``Board of Governors'' means the Board of Governors of the Federal Reserve System.</span>"
</span>  "</span>title-of-act</span>" "</span>H.R.4173 - Dodd-Frank Wall Street Reform and Consumer Protection Act</span>"
</span>  "</span>public-law-number</span>" {"</span>from</span>": </span>111 </span>"</span>to</span>": </span>203</span>} 
</span>  "</span>statute</span>" "</span>1387</span>"
</span>  "</span>year-of-enaction</span>" </span>2010
</span>  "</span>url</span>" "</span>https://www.congress.gov/bill/111th-congress/house-bill/4173/text?r=1</span>"}]
</span></code></pre>
We are going to add one additional feature to improve this representation. Note that depending on what we search for, certain the strings representing the values will vary a lot while those for the keys will remain the same. For example the value for the returned description can be "(3) Board of governors.--The term ``Board of Governors'' means the Board of Governors of the Federal Reserve System."</code> if we search for the term "Board of governors", while it will be "(4) Bureau.--The term ``Bureau'' means the Bureau of Consumer Financial Protection established under title X."</code> if we search for "Bureau". However the key for both of these values would be "description".</p>
The solution in EDN is to use keywords, for the commonly used strings. The names of these are prefaced by :</code> instead of putting them in quotation marks. While JSON does not have such keywords, many other languages do. Notably JSON-LD</a> can use Uniform Resource Identifiers (URI)</a> for a similar purpose.</p>
Using keywords we can give an output for our definition search as follows:</p>
[{ </span>:description </span>"</span>(3) Board of governors.--The term ``Board of Governors'' means the Board of Governors of the Federal Reserve System.</span>"
</span>  </span>:title-of-act </span>"</span>H.R.4173 - Dodd-Frank Wall Street Reform and Consumer Protection Act</span>"
</span>  </span>:public-law-number</span>"</span> {</span>"from"</span>: 111 </span>"to"": </span>203</span>} 
</span>  </span>:statute</span>" "</span>1387</span>"
</span>  :year-of-enaction 2010
</span>  :url </span>"https://www.congress.gov/bill/111th-congress/house-bill/4173/text?r=1"</span>}]
</span></code></pre>
As one can see from just these few examples there can be quite a few ways in which domain concepts are represented. A small set of examples, that describe the elements of the domain in a way that is understandable for both the domain expert and the software engineer can be invaluable. The great benefit of Data Oriented Domain Design, is that it co-opts some battle tested light-weight data representation languages, such as EDN, for this purpose. This allows for some excellent test cases to use when developing the software with methodologies such as Test Driven Development (TDD)</a>.</p>

Of course the above approach has some limitations. </p>
A notation such as JSON-LD</a> can be more expressive with representing the domain and is more capable of representing linked data (i.e.: data that is interconnected with other data). However starting to model the domain with explicit links to other data sources can be more complex especially in cases where domain experts formalize the domain for the first time. As JSON-LD is designed to provide a smooth upgrade path from JSON, starting out with a pure JSON based modelling of the domain can be a great initial step.</p>
DODD also explains the domain through a set of examples, but not through a comprehensive set of restrictions that model the domain in a more complete way. However there exist schema languages, such as Clojure Spec</a> or OWL Ontologies</a> that can model more of such restrictions, and could extend validation a lot further.</p>
Another issue is that these lightweight data languages are often designed for the perspective of a software engineer, that most likely is going to utilize them. However with some tooling they could be made more accessible with domain experts, especially as they are relatively straightforward, with fewer elements, compared to more complex representations.</p>
Nonetheless even with these limitations, Data Oriented Domain Design in which a lightweight data notation language is used to help express examples of a domain to both domain experts and software engineers, can provide a relatively gentle start to modelling the domain through a set of examples. Due to its relative simplicity it could be applied as an initial step, before more "heavyweight" models are brought into the picture. It can provide a great tool for some frank discussions on how a software should function in a particular domain. </p>


The Pharo Prince of Bel-Air
2021-01-24T00:00:00+00:00
Recently I have been aiming to explore the Pharo</a> programming language. The close combination between the language and the Integrated Development Environment (IDE)</a>, makes it quite unlike most other languages that I have encountered. You can, for example, inspect every program element in a visual environment, even on a currently running program. While exploring, I spent much time using the Playground</code> feature, where you easily start executing, and interacting with, small snippets of code.</p>
While exploring the language, the intro-song</a> to the tv-show "The Fresh Prince of Bel-Air</a>" got stuck in my head. The song tells the story of the title character, Will Smith, on how he ended up being "The Fresh Prince of Bel-Air". To paraphrase the song, Will was born and raised in Philadelphia, spending much of his time on the playground. On one occasion, while playing basketball, he got into a fight with some troublemakers. This experience got his mom scared so he was sent off to live with his aunt and uncle in Bel Air. After a flight and a taxi ride, he arrives in Bel Air at his uncles and aunts house, to become "the Prince of Bel-Air".</p>
In this article, let’s explore Pharo together with the story from “The Fresh Prince of Bel Air”. If one wants to follow along, feel free to download</a> and setup Pharo. The language has a really smooth process for getting started, as one can download and install the Pharo Launcher</a> and be pretty much set.</p>
It is difficult to do the language justice by text, because it is so intertwined with the IDE and image based development. I will try to illustrate things as best as I can.</p>
Once we have installed and started the Pharo Launcher the first step is to create a Pharo image for our project. One can think of a Pharo image, as a store of all the objects created by the program. This not only includes what would normally be represented as files of source code of other languages, but also the objects of the running program, as well as the state of the editor. As an example, suppose we create an application for taking notes, and while developing, we create two notes to experiment and play around with. We can now save the Pharo image including these objects. The next time we start the image our notes will be there, alongside with the state of the IDE as we have left it. This can help tremendously when developing or debugging a new application.</p>
To come back to our "The Fresh Prince of Bel-Air" example, first we are going to create an image which will contain our project (see a screenshot of this below). </p>


    
    
    
        
        Creating a Pharo image for our 'The Fresh Prince of Bel-Air' example.</h4>
        
        
    </figcaption>
</figure>

In the Pharo launcher, we can click on the "New" button and get a dialog window for the image that we want to create. We base our image on the last stable Pharo version, which is at the time of this writing version 8.0, enter a name and a short description for our image and click "Create Image".</p>
In a short order our image is then initialized, and we are greeted in our IDE by a welcome message, and some guides and documentation in a menu. I highly recommend the “ProStef” tutorial from this set of guides.</p>
Our next step is the playground which, as mentioned at the start of the article, is a place for experimentation and playing around with code that we are going to develop. One can open this Playground from the menu "Tools" on the top of the screen by selecting "Playground" or by the shortcut CTRL+O+W</code>. </p>
Note that the shortcuts might be different depending on settings and operating systems, as the command key might be either CTRL</code>, CMD</code> or ALT</code>. Thankfully the IDE shows shortcuts very neatly next to the clickable menu options, so they should be easy to figure out and learn.</p>
Once the Playground is open we have a canvas where we can start to write and execute our code. </p>
Before I get into the actual creation of the example I will briefly try to summarize how Pharo as a language works. In Pharo everything is an object</em> and any computation is done by sending messages to objects. Creating such objects is done through specific objects known as classes</em>, which define structure and behaviour of the objects they create. </p>
These classes are organized into hierarchies. Hierarchies allow you to reuse behaviour. A class lower in the hierarchy, the so-called subclass</em> can inherit the behaviour of a class higher in the hierarchy, called a superclass</em>. At the root of this hierarchy is the class named Object</em>, which ensures as we noted that anything we create is by definition and behaviour an Object</em>.</p>
We call the objects created by the class as their instances</em>. The methods</em> of an object specify how the messages an object receives should be handled. When an object receives a message</em> it aims to find the method that has the same name as the message to handle it. An object</em> also encapsulates data, such as dates, numbers, strings and other objects, based on what the object represents into variables. It can use this data to help handle the messages it receives. </p>
An object can refer to itself by using the special identifier self</em>. In order to create a new object of a class, the initialize</em> message needs to be sent which gets handled by the initialize</em> method, though often the message new</em> is used as a shorthand for this.</p>
For people with some experience with another Object Oriented programming language</a> the above description should sound familiar. One thing that stands out for me on a first glance is how consistently the use of objects and message passing is applied within Pharo. Everything</em> is an object and interaction seems to always happen by message passing. The above summary should hopefully be enough to follow the examples below but for a more in depth look I highly recommend the Learning Object-Oriented Programming, Design with TDD in Pharo</a> book.</p>

Now given the above description of Pharo, we want to apply it by creating a representation of what happens in the intro song in "The Fresh Prince of Bel-Air" by sending messages. In particular we want to represent the character of "Will Smith" in the intro as an object and the changes in location and activity based on the intro song. </p>
So let's start off by creating a new Object representing Will. We aim to do this by instantiating a class called FPWillSmith</code> by typing the following into the Playground.</p>
will </span>:</span>= FPWillSmith new.
</span></code></pre>
Here we want to send the New</em> message to the class FPWillSmith</code> to create a new object of this class and assign it to the variable will</code>. Let us quickly go over the syntax elements for this statement. </p>
The operator :=</code> is used in Pharo for assignment, and in this case to assign our new object instance to the will</code> variable. We also send to class FPWillSmith</code>, functioning as the receiver, the message new</code> to create a new instance of it. Messages like this that are sent to an object without any additional information are called unary</code> messages. These are always in the form of receiver message</code>. Lastly, statements in Pharo are separated/ended by a .</code>. This can be omitted in many cases but we keep it around in our examples to show its use.</p>
To execute this statement we can right click it and select 'Do It' or use the short-cut for this CTRL+D</code>.</p>
After doing this we will get the following dialog:</p>

    
    
    
        
        We did not define what we mean by FPWillSmith which brings us some options on what to do now.</h4>
        
        
    </figcaption>
</figure>

Well, whoops it seems we kind of forgot to define our class FPWillSmith</code>, so we get a notification for it. The reason why I am highlighting this error is to show that unlike many other languages, where you would just get an error at either compile or runtime, the system brings up a dialog to help us solve this issue. The only other language where I experienced a similar dialog was with CommonLisp</a>.</p>
So lets click "Define a new class" and give a class definition as follows:</p>

    
    
    
        
        We define a class FPWillSmith, to ensure that every object of this class holds some data about the object's name, location and activity.</h4>
        
        
    </figcaption>
</figure>

A few things to note for this code in this picture: </p>
Object </span>subclass:</span> #FPWillSmith
</span>       </span>instanceVariableNames: </span>'name </span>location activity</span>' 
</span>       </span>classVariableNames: </span>''
</span>       </span>category:</span> FreshPrinceDemo
</span></code></pre>
Much like every other part of Pharo, the creation of a new class also happens through message passing. Here a keyword message is sent to the Object</code> class to define our new class. Unlike unary messages, these messages use one or more keywords to send additional object information along with the message. These are defined in the form of receiver keyword1: object1 keyword2: object2 ...</code>  where the keyword and object pairs make up the message to the receiving object.</p>
We will go over the objects sent in the message one by one. The subclass</code> part of the message defines the name of our new class FPWillSmith</code>. The #</code> in front of FPWillSmith</code> indicates a symbol that is globally unique, unlike a string, which is part of the requirement for this subclass message. Next is defining the name of the instance variables using the keyword instanceVariablesNames</code>, that will represent the elements of an instance of FPWillSmith</code> that we would like to use. There are also classVariableNames</code> that describe such elements for the class itself as opposed to the instances, but here we do not define any. Finally, there is a category</code> defined as FreshPrinceDemo</code>, that is essentially the name of the package that we are currently defining.</p>
Now that we have created our FPWillSmith</code> class, we can now take our original statement:</p>
will </span>:</span>= FPWillSmith new.
</span></code></pre>
and execute it, by right clicking it and selecting 'Do It', or using a shortcut such as CTRL+D</code>. As you can see, we now don’t receive errors.</p>
What we would like to do is to double check, whether the object we created is what we want. Pharo has a really neat feature for inspecting any</em> object. We can right-click this statement in the playground and select 'Inspect It' from the menu, or using the shortcut CTRL+I</code> for giving us the object inspector. This should give us a window such as this:</p>

    
    
    
        
        Inspecting our newly created object.</h4>
        
        
    </figcaption>
</figure>

As we can see from the inspector, while the object has all the instance variables that we wanted (name, location, activity), the value for this is nil</code> which is not what we want.</p>
In order to make sure there are meaningful values for these variables for a new object, we need to define our own initialize</code> method that will get called for every new object.</p>
First we are going to look up our class in the code browser. This can be done by clicking the menu options Tools and then Systems Browser, or by using the short-cut CTRL+O+B</code>. </p>

    
    
    
        
        The System Browser, where we filtered for our new Package and selected our `FPWillSmith` class , showing the template for a new method.</h4>
        
        
    </figcaption>
</figure>

The system browser can be used to find, organize and edit classes and methods for all</em> packages within the Pharo image. It is divided up into four columns. The leftmost column lists the packages. We can use this to filter for our new package named 'FreshPrinceDemo'. The second column on the left details the classes in the selected package. For our new package we should only have one class here: FPWillSmith</code>. The third column lists all the protocols</em> of the selected class. Protocols are a way to group together related methods, which makes for easier browsing and search. Finally the fourth column from the left lists all the methods, for the given class or selected protocol. </p>
The panel below these four columns is also very interesting. If one selects a package, it shows in a tab a template for defining a class. If one selects a class, it shows a tab for a template on how to define new methods. It also has a tab for the class definition itself .</p>
In order to proceed let's use the method definition template to define our new constructor, as follows:</p>
initialize
</span>	name </span>:</span>= </span>'Will </span>Smith</span>'.
</span>	location </span>:</span>= </span>'West </span>Philadelphia playground</span>'.
</span>	activity </span>:</span>= </span>'Playing </span>basketball</span>'.
</span></code></pre>
which we can fill into the new instance side method window below, instead of the template code.</p>
What this definition does is to create a constructor, that sets the values for name, location and activity to values that makes sense, given where Will is and what he is doing in the beginning of the "The Fresh Prince of Bel-Air" intro song.</p>
Make sure to accept these values, using right click on the panel and selecting Accept or by pressing the shortcut CTRL+S</code> (you might also need to save your name as the author for this piece of code in a dialog along the way).</p>
We will now have the initialization method defined:</p>

    
    
    
        
        The System Browser showing our newly defined constructor method.</h4>
        
        
    </figcaption>
</figure>

We can check whether our new constructor has worked by inspecting the statement we initially created in the Playground:</p>

    
    
    
        
        Inspecting the new instance after saving our new constructor.</h4>
        
        
    </figcaption>
</figure>

Now the will</code> representation of the character has the right name, location and activity, but we want to express a bit more of the intro song. We will do this by sending messages to the will</code> object after which we expect that the location and the activity changes accordingly.</p>
Let's assume we instead have the following in the Playground:</p>
will </span>:</span>= FPWillSmith new.
</span>will getsInTrouble.
</span></code></pre>
As we can now expect, executing ('Doing It') of the second statement will not go well as we did not yet define a way for the object to handle this message. In fact this message will bring up the debugger because the object does not know how to handle the message getsInTrouble</code>. </p>

    
    
    
        
        We go into the Debugger when the object can not handle the getsInTrouble message.</h4>
        
        
    </figcaption>
</figure>

We can fix this by defining a method, named getsInTrouble</code>. We can even do this from the debugger by clicking on the Create</code> button at the top of the debugger to define getsInTrouble</code> method for the FPWillSmith</code> class. After getting a popup for picking a protocol, which we do not need to do for this tutorial, we get the window showing a placeholder definition for the method:</p>
getsInTrouble
</span>	</span>self</span> shouldBeImplemented.
</span></code></pre>
We want to replace this with the following definition, that updates the activity of the object:</p>
getsInTrouble
</span>	activity </span>:</span>= </span>'In </span>trouble</span>'.
</span></code></pre>

    
    
    
        
        The getsInTrouble method defined in the debugger, after which we can proceed, with the tool-tip of the proceed button highlighted.</h4>
        
        
    </figcaption>
</figure>

We have now created the method from inside the debugger, while the execution of the code was put on hold! With the definition saved we can even click the Proceed</code> button on the top of the debugger to continue from where we halted.</p>
Now the code correctly executes and when we next inspect the last statement have the activity properly updated.</p>

    
    
    
        
        The activity now properly updates on the object after receiving the getsInTrouble message.</h4>
        
        
    </figcaption>
</figure>

For the final piece we will define a method for sending the object to aunt and uncle, with the following definition:</p>
sentToAuntAndUncle
</span>	activity </span>:</span>= </span>'Being </span>the Prince of Bel-Air</span>'.
</span>	location </span>:</span>= </span>'Bel-Air'.
</span></code></pre>
If one follows along, we will leave it up to you on where to define this method from, the debugger or the system browser. </p>
Once we have defined it we can put the following in the Playground:</p>
will </span>:</span>= FPWillSmith new.
</span>will getsInTrouble.
</span>will sentToAuntAndUncle.
</span></code></pre>
and inspect our object:</p>

    
    
    
        
        The object now shows the new updated activity as 'Being the Prince of Bel-Air' in the location 'Bel-Air'.</h4>
        
        
    </figcaption>
</figure>

That is it for this short tutorial. I hope I have shown a small glimpse of why Pharo is such an interesting language.</p>
We have started with no definitions at all, and slowly built up what we wanted to do in a very organic way. We just started putting the statements down in the Playground for passing messages to objects, and created the definitions as we went along. Whenever something has given an error in the debugger or did not show up to our liking in the inspector, we slowly added more code. In the process we have evolved an object that initially represented no aspect of Will's character, to something that had a name, location and activity, including; “Playing basketball” in a “West Philadelphia playground”. </p>
And at the end we are now finally there,
having the object's activity denoted as Being the Prince of Bel-Air</code>.</p>


Meditations in your Pocket
2020-12-28T00:00:00+00:00
Meditations</a> are a series of notes that the Roman emperor Marcus Aurelius</a>
wrote for guidance and self-improvement. While it is likely that the emperor wrote them purely for his own benefit, they have been published after his death and contain elements of Stoic philosophy</a> that many people have consulted since.</p>
One way achieve a bit of self-improvement, coding-wise, is to pick a small project to do with a new set of tools. In particular, I have found the Flutter</a> framework for creating mobile apps very interesting. Unfortunately I did not have the opportunity to explore Flutter much beyond running some sample apps</a> previously. To bring a change to this I thought it would be a nice small project to create a small app for browsing the text of "Meditations" in Flutter. I aim to show what I have built in this article, organize my thoughts, and hopefully help others unfamiliar with Flutter to get a small glimpse of what it can do.</p>
To give a preview of the implemented functionality, here is an animation of the functionality of the app:</p>

    
    
    
        
        Navigating the text of Meditations in the app running on an Android emulator.</h4>
        
        
    </figcaption>
</figure>

The full code of the application presented in this article is available</a> to explore. If one wants to follow along by running the code, or has a similar project in mind, the first step is to install Flutter</a>. There are quite a few dependencies needed for mobile development, but thankfully there is some excellent documentation</a> available on how to get started from the official Flutter site. For code editing I highly recommend Visual Studio Code</a> with the Flutter plugin, but other options are also available</a>.</p>
If everything is installed correctly the flutter</code> command line tool will be available. For this project I intended to create an Android app, but the framework allows for app development for iOS (Apple) phones, as well as desktop and web targets. For Android development, aside from the development toolchain, it is probably good idea to have either an emulator or a device for Android development setup and connected. This allows us to the result of the code up and running. In fact, on of the biggest draws of the Flutter framework is Hot Reload</a>, which allows on to make changes and quickly see the results in the running app, without reinstalling everything from scratch.</p>
To get started with a fresh project, the command flutter create</code> will create a directory with all the necessary files, while flutter run</code> will ensure that the project is run on the emulator or connected device for testing, debugging and seeing our work. For the "Meditations" app, we could start by creating a new project named Meditations</code> and making changes from there on.</p>
The default generated project will create a bunch of files and directories required for an example app, which is a very small application that has a single button and a screen that shows how many times that button has been pressed (see the test drive</a> part of the Flutter docs). The core part are the files main.dart</code> and widget_test.dart</code>, in the lib</code> and test</code> directories respectively. The former is responsible for defining and running the app, while the later exists for testing the functionality.</p>
In order to get the Meditations app up and running, we will have four files in total. Similarly to the default project we have a main.dart</code> and a widget_test.dart</code> for defining and testing our app, but with different contents. In addition, we also have a meditations.dart</code> file where we define how to search and navigate the text of the Meditations, as well as the file meditations_test.dart</code> to test this search and navigation functionality. </p>
Although we do not have to go full Test Driven Development (TDD)</a> for such a small demo application, it is a good idea and practice to start off writing some tests to define what behavior we want to show.</p>
We first start describing our application by walking through (some of) the contents of meditations_test.dart</code>. It also gives us an opportunity to ease us into the Dart language</a> that the Flutter framework uses, in case if the reader is unfamiliar with it. It would be too much for this short article to fully explain the Dart language, for which there are some excellent guides</a>, but I hope I can make the functionality clear enough, even for readers completely new to it.</p>
import </span>'package:meditations/mediations.dart'</span>;
</span>import </span>'package:test/test.dart'</span>;
</span></code></pre>
The first portion is importing elements that we are going to test, to be defined in the mediations.dart</code> file, and functionality for defining tests, which is in the test</code> package.</p>
Next up are the tests that are in the main</code> method. As in some other languages, the main method is where the app execution starts. In the case of testing, the tests within the 'main' method are executed. The tests themselves contain information about the purpose of the test, the returned value (or values) of the functionality under test and the expected value of what we believe should be returned.</p>
Below are some examples of such tests:</p>
  </span>test</span>(</span>'Checking the existance of a Book that exists.'</span>, () {
</span>    </span>expect</span>(</span>Meditations</span>.</span>existBook</span>(</span>0</span>), </span>true</span>);
</span>  });
</span>
</span>  </span>test</span>(</span>'Checking the existance of a Book that can not exist.'</span>, () {
</span>    </span>expect</span>(</span>Meditations</span>.</span>existBook</span>(-</span>1</span>), </span>false</span>);
</span>  });
</span>
</span>  </span>test</span>(</span>'Checking the existance of a Book that does not exist.'</span>, () {
</span>    </span>expect</span>(</span>Meditations</span>.</span>existBook</span>(</span>40</span>), </span>false</span>);
</span>  });
</span>
</span>  </span>test</span>(</span>'Checking the existance of a Book Section that exists.'</span>, () {
</span>    </span>expect</span>(</span>Meditations</span>.</span>existBookSection</span>(</span>0</span>, </span>1</span>), </span>true</span>);
</span>  });
</span>
</span>  </span>test</span>(</span>'Checking the existance of a Book Section that can not exist.'</span>, () {
</span>    </span>expect</span>(</span>Meditations</span>.</span>existBookSection</span>(</span>0</span>, -</span>10</span>), </span>false</span>);
</span>  });
</span>
</span>  </span>test</span>(</span>'Checking the existance of a Book Section in a non-existent Book.'</span>, () {
</span>    </span>expect</span>(</span>Meditations</span>.</span>existBookSection</span>(</span>40</span>, </span>0</span>), </span>false</span>);
</span>  });
</span>
</span>  </span>test</span>(
</span>      </span>'Checking the existance of a Book Section that does not exist in an existing Book.'</span>,
</span>      () {
</span>    </span>expect</span>(</span>Meditations</span>.</span>existBookSection</span>(</span>0</span>, </span>40</span>), </span>false</span>);
</span>  });
</span></code></pre>
In order to understand what these tests want to achieve, first we need to dive deeper about what functionality that is implemented in Meditations.dart</code> which we intend to test. The purpose of it is to easily find a particular part in the text of 'Meditations'. 'Meditations' itself is divided into a total of 12 books, each with multiple sections of text. Instead of showing the full text of all books on a single screen we want to make the user able to navigate between the text of each of these sections. </p>
Part of implementing this navigation process requires having the ability to check, for a given book number, whether that book exists. As in Dart indexes start at 0, the method existBook</code> should return true</code> for any integer from 0 to 11, and false otherwise. Later on we will show how we will turn the book and section numbers back to 1-indexed ones in the UI of the app. Similarly to checking the existence of a book, we want to able to check if a specific section in a specific book exists, which the method existBookSection</code> should provide.</p>
Of course the functionality should not end here. In particular we also want to be able to get the text of the first and last sections of Meditations. In order to keep the return of these search results simple, we also return the book and section numbers (0-indexed) along with the text where possible as well, even though for the first section, we should already know the book and section numbers. </p>
As Dart is an object oriented language, we create a specific object of the class BookSectionText</code> to hold these elements (more on the definition of this class a bit later).</p>
  </span>test</span>(</span>'Getting the first part of Meditations returns Book 1, Section 1.'</span>, () {
</span>    </span>BookSectionText</span> bst = </span>Meditations</span>.</span>getFirst</span>();
</span>    </span>var</span> firstText = </span>Meditations</span>.book1.first;
</span>    </span>var</span> expectedBookNr = </span>0</span>;
</span>    </span>var</span> expectedSectionNr = </span>0</span>;
</span>    </span>expect</span>(bst.bookNr, expectedBookNr);
</span>    </span>expect</span>(bst.sectionNr, expectedSectionNr);
</span>    </span>expect</span>(bst.text, firstText);
</span>  });
</span>
</span>  </span>test</span>(
</span>      </span>'Getting the last part of Meditations returns last Section of the last Book.'</span>,
</span>      () {
</span>    </span>BookSectionText</span> bst = </span>Meditations</span>.</span>getLast</span>();
</span>    </span>var</span> lastText = </span>Meditations</span>.books.last.last;
</span>    </span>var</span> expectedBookNr = </span>Meditations</span>.books.length - </span>1</span>;
</span>    </span>var</span> expectedSectionNr = </span>Meditations</span>.books.last.length - </span>1</span>;
</span>    </span>expect</span>(bst.bookNr, expectedBookNr);
</span>    </span>expect</span>(bst.sectionNr, expectedSectionNr);
</span>    </span>expect</span>(bst.text, lastText);
</span>  });
</span></code></pre>
The final set of tests we are going over are for the methods for finding the previous section and the next one. This seems generally a straightforward thing to do, when moving to the previous or next section in the same book. However for convenience we also want to move to the next, or previous, book if the these are unavailable in the current book, but exist in another one. So the functions for nextSection</code> and previousSection</code> has to have this 'roll-over' functionality included which will make for a nicer user interface. However even with roll-over, there are cases when we ran out of sections to show (e.g.: when wanting to get the next section after the last one). In this case we use an object of the class NoSuchBookSection</code> to indicate this.</p>
Below are some tests for the nextSection</code> and previousSection</code> functionality:</p>
  </span>test</span>(</span>'Getting the next Section of a Book.'</span>, () {
</span>    </span>BookSectionText</span> bst = </span>Meditations</span>.</span>getNextSection</span>(</span>11</span>, </span>3</span>);
</span>    </span>var</span> expectedBookNr = </span>11</span>;
</span>    </span>var</span> expectedSectionNr = </span>4</span>;
</span>    </span>var</span> expectedText = </span>Meditations</span>.books[expectedBookNr][expectedSectionNr];
</span>    </span>expect</span>(bst.bookNr, expectedBookNr);
</span>    </span>expect</span>(bst.sectionNr, expectedSectionNr);
</span>    </span>expect</span>(bst.text, expectedText);
</span>  });
</span>
</span>  </span>test</span>(</span>'Getting the next Section of a Book when not available.'</span>, () {
</span>    </span>BookSectionSearchResult</span> bst = </span>Meditations</span>.</span>getNextSection</span>(</span>12</span>, </span>30</span>);
</span>    </span>expect</span>(bst is </span>NoSuchBookSection</span>, </span>true</span>);
</span>  });
</span>
</span>  </span>test</span>(</span>'Getting the next Section in the next Book'</span>, () {
</span>    </span>BookSectionText</span> bst = </span>Meditations</span>.</span>getNextSection</span>(</span>0</span>, </span>16</span>);
</span>    </span>var</span> expectedBookNr = </span>1</span>;
</span>    </span>var</span> expectedSectionNr = </span>0</span>;
</span>    </span>var</span> expectedText = </span>Meditations</span>.books[expectedBookNr][expectedSectionNr];
</span>    </span>expect</span>(bst.bookNr, expectedBookNr);
</span>    </span>expect</span>(bst.sectionNr, expectedSectionNr);
</span>    </span>expect</span>(bst.text, expectedText);
</span>  });
</span>
</span>  </span>test</span>(</span>'Getting the previous Section of a Book.'</span>, () {
</span>    </span>BookSectionText</span> bst = </span>Meditations</span>.</span>getPreviousSection</span>(</span>0</span>, </span>3</span>);
</span>    </span>var</span> expectedBookNr = </span>0</span>;
</span>    </span>var</span> expectedSectionNr = </span>2</span>;
</span>    </span>var</span> expectedText = </span>Meditations</span>.books[expectedBookNr][expectedSectionNr];
</span>    </span>expect</span>(bst.bookNr, expectedBookNr);
</span>    </span>expect</span>(bst.sectionNr, expectedSectionNr);
</span>    </span>expect</span>(bst.text, expectedText);
</span>  });
</span>
</span>  </span>test</span>(</span>'Getting the previous Section of a Book when not available.'</span>, () {
</span>    </span>BookSectionSearchResult</span> bst = </span>Meditations</span>.</span>getPreviousSection</span>(</span>0</span>, </span>0</span>);
</span>    </span>expect</span>(bst is </span>NoSuchBookSection</span>, </span>true</span>);
</span>  });
</span>
</span>  </span>test</span>(</span>'Getting the previous Section in the previous Book'</span>, () {
</span>    </span>BookSectionText</span> bst = </span>Meditations</span>.</span>getPreviousSection</span>(</span>1</span>, </span>0</span>);
</span>    </span>var</span> expectedBookNr = </span>0</span>;
</span>    </span>var</span> expectedSectionNr = </span>16</span>;
</span>    </span>var</span> expectedText = </span>Meditations</span>.books[expectedBookNr][expectedSectionNr];
</span>    </span>expect</span>(bst.bookNr, expectedBookNr);
</span>    </span>expect</span>(bst.sectionNr, expectedSectionNr);
</span>    </span>expect</span>(bst.text, expectedText);
</span>  });
</span>}
</span></code></pre>
Now that we got the tests for the core functionality of navigating the text of Meditations done. Next we will go over the actual implementation of what we aim to test in the file meditations.dart</code>.</p>
First off are the class definitions for the classes that we mentioned above:</p>
class </span>BookSectionSearchResult</span> {}
</span>
</span>// This represents the result of search, which results in a text with the attached book and section number.
</span>class </span>BookSectionText </span>extends </span>BookSectionSearchResult</span> {
</span>  </span>int</span> bookNr;
</span>  </span>int</span> sectionNr;
</span>  </span>String</span> text;
</span>}
</span>
</span>// This represent the result of a search, in which case no section is found.
</span>class </span>NoSuchBookSection </span>extends </span>BookSectionSearchResult</span> {}
</span></code></pre>
If the reader is familiar with an object oriented language, the definitions should be reasonably straightforward. The search results are represented by two types of classes that we used in our tests. An instance of a class NoSuchBookSection</code> represents that no valid text could be returned and an instance BookSectionText</code> represents the search results that returned the text of a specific section and its book/section numbers. These are both subclasses of the class BookSectionSearchResult</code> which allows us easily define the return type of the methods getPreviousSection</code> and getNextSection</code>.</p>
Also note that the code here contains single line comments which are denoted after a //</code> in Dart.</p>
Next up is the definition of the Meditations</code> class that describes the text of Meditations by Marcus Aurelius and how to access them.</p>
It starts with the snippet:</p>
class </span>Meditations</span> {
</span>  </span>static final</span> book1 = [
</span>    </span>'''I. Of my grandfather Verus I have learned to be gentle and meek, ... 
</span>    // End of snippet. 
</span></code></pre>
Of course quoting the full text of the representation is too much for this article. Even in the code of this demo app</a> we only represent the full text of the first and last books in detail at the moment, although the structure to implement every section exists. </p>
However we can say we represent the text as a list of books, where each book is itself a list of sections which are represented by the text of these sections.</p>
In Dart, we can represent a list as a set of items in between square brackets, i.e. []</code>, such as follows:</p>
  </span>static final</span> books = [
</span>    book1,
</span>    book2,
</span>    book3,
</span>    book4,
</span>    book5,
</span>    book6,
</span>    book7,
</span>    book8,
</span>    book9,
</span>    book10,
</span>    book11,
</span>    book12
</span>  ];
</span></code></pre>
The other interesting part is that we use the keywords static</code> and final</code>. The static</code> keyword indicates that the variables are class-wide, as the text in these books would be same for any instance of the Meditations</code> class. The final</code> keyword indicates that we do not intend to change these variables once they are defined, as the text of Meditations will not be changing in our app. </p>
The final piece of our Meditations</code> class are methods for accessing and navigating the texts that we described previously in our tests, such as getFirst</code>, getLast</code>, getPreviousSection</code> and getNextSection</code>. Hopefully following them should be reasonably straightforward given the comments and our tests. The only additional thing to note that these methods are also marked as static</code>. As the Meditations</code> class does not contain any variables that would change per instance, every one of these methods could be made available at a class level (e.g. it could be called as Meditations.getNextSection(11, 3)</code>). </p>
See the text of these methods below:</p>
// We assume that there is always at least one book in the list of books and at least one section in each book with a text.
</span>
</span>  </span>// Checks if the book with given book number exists.
</span>  </span>static </span>bool </span>existBook</span>(</span>int</span> bookNr) {
</span>    </span>if</span> (bookNr < </span>0 </span>|| bookNr > books.length - </span>1</span>) {
</span>      </span>return </span>false</span>;
</span>    } </span>else</span> {
</span>      </span>return </span>true</span>;
</span>    }
</span>  }
</span>
</span>  </span>// Checks if the section with given book and section numbers exists.
</span>  </span>static </span>bool </span>existBookSection</span>(</span>int</span> bookNr, </span>int</span> sectionNr) {
</span>    </span>if</span> (!</span>existBook</span>(bookNr)) {
</span>      </span>return </span>false</span>;
</span>    } </span>else</span> {
</span>      </span>var</span> selectedBook = books[bookNr];
</span>      </span>if</span> (sectionNr < </span>0 </span>|| sectionNr > selectedBook.length - </span>1</span>) {
</span>        </span>return </span>false</span>;
</span>      } </span>else</span> {
</span>        </span>return </span>true</span>;
</span>      }
</span>    }
</span>  }
</span>
</span>  </span>// Returns the length of a book, in the number of sections.
</span>  </span>static </span>int </span>bookLength</span>(</span>int</span> bookNr) {
</span>    </span>if</span> (!</span>existBook</span>(bookNr)) {
</span>      </span>return </span>0</span>;
</span>    } </span>else</span> {
</span>      </span>var</span> selectedBook = books[bookNr];
</span>      </span>return</span> selectedBook.length;
</span>    }
</span>  }
</span>
</span>  </span>// Returns the search result for a given book and section number.
</span>  </span>static </span>BookSectionSearchResult </span>getText</span>(</span>int</span> bookNr, </span>int</span> sectionNr) {
</span>    </span>if</span> (!</span>existBookSection</span>(bookNr, sectionNr)) {
</span>      </span>return </span>NoSuchBookSection</span>();
</span>    } </span>else</span> {
</span>      </span>var</span> bst = </span>new </span>BookSectionText</span>();
</span>      bst.bookNr = bookNr;
</span>      bst.sectionNr = sectionNr;
</span>      bst.text = books[bookNr][sectionNr];
</span>      </span>return</span> bst;
</span>    }
</span>  }
</span>
</span>  </span>// Returns the first section.
</span>  </span>static </span>BookSectionSearchResult </span>getFirst</span>() {
</span>    </span>var</span> selectedBook = books[</span>0</span>];
</span>    </span>if</span> (selectedBook == </span>null</span>) {
</span>      </span>return </span>NoSuchBookSection</span>();
</span>    }
</span>    </span>var</span> selectedSection = selectedBook[</span>0</span>];
</span>    </span>if</span> (selectedSection == </span>null</span>) {
</span>      </span>return </span>NoSuchBookSection</span>();
</span>    } </span>else</span> {
</span>      </span>var</span> bst = </span>new </span>BookSectionText</span>();
</span>      bst.bookNr = </span>0</span>;
</span>      bst.sectionNr = </span>0</span>;
</span>      bst.text = selectedSection;
</span>      </span>return</span> bst;
</span>    }
</span>  }
</span>
</span>  </span>// Returns the last section.
</span>  </span>static </span>BookSectionSearchResult </span>getLast</span>() {
</span>    </span>var</span> selectedBook = books.last;
</span>    </span>if</span> (selectedBook == </span>null</span>) {
</span>      </span>return </span>NoSuchBookSection</span>();
</span>    }
</span>    </span>var</span> selectedSection = selectedBook.last;
</span>    </span>if</span> (selectedSection == </span>null</span>) {
</span>      </span>return </span>NoSuchBookSection</span>();
</span>    } </span>else</span> {
</span>      </span>var</span> bst = </span>new </span>BookSectionText</span>();
</span>      bst.bookNr = </span>Meditations</span>.books.length - </span>1</span>;
</span>      bst.sectionNr = </span>Meditations</span>.books.last.length - </span>1</span>;
</span>      bst.text = selectedSection;
</span>      </span>return</span> bst;
</span>    }
</span>  }
</span>
</span>  </span>// Given a book- and a section number, gets the next section.
</span>  </span>static </span>BookSectionSearchResult </span>getNextSection</span>(</span>int</span> bookNr, </span>int</span> sectionNr) {
</span>    </span>//Get the next section in the current book
</span>    </span>var</span> nextSection = </span>getText</span>(bookNr, sectionNr + </span>1</span>);
</span>    </span>//If it exits return it
</span>    </span>if</span> (!(nextSection is </span>NoSuchBookSection</span>)) {
</span>      </span>return</span> nextSection;
</span>    } </span>else</span> {
</span>      </span>//Otherwise get the first section of the next book
</span>      </span>return </span>getText</span>(bookNr + </span>1</span>, </span>0</span>);
</span>    }
</span>  }
</span>
</span>  </span>// Given a book- and a section number, gets the previous section.
</span>  </span>static </span>BookSectionSearchResult </span>getPreviousSection</span>(</span>int</span> bookNr, </span>int</span> sectionNr) {
</span>    </span>//Get the previous section in the current book
</span>    </span>var</span> previousSection = </span>getText</span>(bookNr, sectionNr - </span>1</span>);
</span>    </span>//If it exits return it
</span>    </span>if</span> (!(previousSection is </span>NoSuchBookSection</span>)) {
</span>      </span>return</span> previousSection;
</span>    } </span>else</span> {
</span>      </span>//Otherwise get the last section of the previous book
</span>      </span>var</span> previousBookNr = bookNr - </span>1</span>;
</span>      </span>return </span>getText</span>(previousBookNr, </span>bookLength</span>(previousBookNr) - </span>1</span>);
</span>    }
</span>  }
</span>}
</span></code></pre>
We have now covered two of the four main files responsible for the functionality of our app, notably the functionality to navigate the text of Meditations and the tests for them. The purpose of the other two is to create the app using this functionality for navigating the text with the Flutter framework (main.dart</code>) and the tests for the app (widget_test.dart</code>). </p>
Let us start with the test first. One of the nice features of the Flutter framework is that one can test the full app nearly as easily than any other Dart code. </p>
First we start importing the functionality for testing, as well as our main app that we aim to test:</p>
import </span>'package:flutter/material.dart'</span>;
</span>import </span>'package:flutter_test/flutter_test.dart'</span>;
</span>import </span>'package:meditations/main.dart'</span>;
</span></code></pre>
Next up is the main method with the functionality that we aim to test. This functionality is similar to the functionality that we have shown previously of our app, seen here again below:</p>
{{< figure src="/img/post/2020/meditations/meditationsapp.gif" title="Navigating the text of Meditations in the app running on an Android emulator." >}}</p>
In particular when we are navigating the text of Meditations, we tap the icons for first, previous, next and last in the bottom navigation bar. </p>
On the screen we not only navigate to the right text, but we also show the number of the book and section of the text that we are currently reading. 
We use this information to create a navigation test, as once the right icon has been tapped, we should be able to predict and check the book and section numbers that should now be on screen. </p>
The test is all tied together with the test functionality of Flutter for setting up the widgets (the elements of interaction) of the app for testing, redrawing the elements after a (simulated) interaction and checking them (more documentation on testing Flutter widgets can be found in the Flutter documentation</a>). The code of this for the app can be seen as below.</p>
void </span>main</span>() {
</span>  </span>testWidgets</span>(</span>'Bottom navigation smoke test'</span>, (</span>WidgetTester</span> tester) </span>async</span> {
</span>    </span>// Build the app and trigger a frame.
</span>    </span>await</span> tester.</span>pumpWidget</span>(</span>MyApp</span>());
</span>
</span>    </span>// Verify that the title shows Book 1 and Section 1.
</span>    </span>expect</span>(find.</span>text</span>(</span>'Meditations Book 1 Section 1'</span>), findsOneWidget);
</span>
</span>    </span>// Tap the icon for next and trigger a frame.
</span>    </span>await</span> tester.</span>tap</span>(find.</span>byIcon</span>(</span>Icons</span>.navigate_next));
</span>    </span>await</span> tester.</span>pump</span>();
</span>
</span>    </span>// Verify that the title has changed correctly.
</span>    </span>expect</span>(find.</span>text</span>(</span>'Meditations Book 1 Section 2'</span>), findsOneWidget);
</span>    </span>expect</span>(find.</span>text</span>(</span>'Meditations Book 1 Section 1'</span>), findsNothing);
</span>
</span>    </span>// Tap the icon for last and trigger a frame.
</span>    </span>await</span> tester.</span>tap</span>(find.</span>byIcon</span>(</span>Icons</span>.last_page));
</span>    </span>await</span> tester.</span>pump</span>();
</span>
</span>    </span>// Verify that the title has changed correctly.
</span>    </span>expect</span>(find.</span>text</span>(</span>'Meditations Book 12 Section 26'</span>), findsOneWidget);
</span>    </span>expect</span>(find.</span>text</span>(</span>'Meditations Book 1 Section 2'</span>), findsNothing);
</span>
</span>    </span>// Tap the icon for previous and trigger a frame.
</span>    </span>await</span> tester.</span>tap</span>(find.</span>byIcon</span>(</span>Icons</span>.navigate_before));
</span>    </span>await</span> tester.</span>pump</span>();
</span>
</span>    </span>// Verify that the title has changed correctly.
</span>    </span>expect</span>(find.</span>text</span>(</span>'Meditations Book 12 Section 25'</span>), findsOneWidget);
</span>    </span>expect</span>(find.</span>text</span>(</span>'Meditations Book 12 Section 26'</span>), findsNothing);
</span>
</span>    </span>// Tap the icon for first and trigger a frame.
</span>    </span>await</span> tester.</span>tap</span>(find.</span>byIcon</span>(</span>Icons</span>.first_page));
</span>    </span>await</span> tester.</span>pump</span>();
</span>
</span>    </span>// Verify that the title has changed correctly.
</span>    </span>expect</span>(find.</span>text</span>(</span>'Meditations Book 1 Section 1'</span>), findsOneWidget);
</span>    </span>expect</span>(find.</span>text</span>(</span>'Meditations Book 12 Section 25'</span>), findsNothing);
</span>  });
</span>}
</span></code></pre>
Now all that remains is to show how the actual app is put together. We will go through the code of it here, found in main.dart</code>, step by step. </p>
First we start with the imports of the widget library we are using as well as our implementation of accessing the text of Meditations.</p>
import </span>'package:flutter/material.dart'</span>;
</span>import </span>'mediations.dart'</span>;
</span></code></pre>
Next is, the very short main method, that ensures the app we are building is run when requested.</p>
void </span>main</span>() {
</span>  </span>// The main method runs the app that was described
</span>  </span>runApp</span>(</span>MyApp</span>());
</span>}
</span></code></pre>
As mentioned in our description of the test, this app consists of a tree of widgets</a> (elements of interaction), the root of which we identify with MyApp</code> in the code. This MyApp widget is the one that gets run by the main method seen above.</p>
The widgets themselves have properties that contain the elements that the widget is using. For example the title of the app, the theme of the app and the other widgets it uses such as a widget for a home page. The main job of the widget is to implement a specific build function that describes how these various elements fit together. In the case of MyApp</code> we are creating a MaterialApp</a> customized to our specific needs. This includes a home page for navigating the text of Mediations, as well as a theme that uses a color (hopefully) close enough to Imperial purple</a>.</p>
class </span>MyApp </span>extends </span>StatelessWidget</span> {
</span>  </span>// The app consists of a tree of widgets, of which this is the root.
</span>  </span>@override
</span>  </span>Widget </span>build</span>(</span>BuildContext</span> context) {
</span>    </span>return </span>MaterialApp</span>(
</span>      title: </span>'Meditations'</span>,
</span>      theme: </span>ThemeData</span>(
</span>        primarySwatch: </span>Colors</span>.purple,
</span>        visualDensity: </span>VisualDensity</span>.adaptivePlatformDensity,
</span>      ),
</span>      home: </span>MyHomePage</span>(title: </span>'Meditations'</span>),
</span>    );
</span>  }
</span>}
</span></code></pre>
The previous Widget does not carry an internal, mutable, state directly in itself (hence why it is an extension of a StatelessWidget</a>). However, as one can deduce from our tests, we want to keep information such as the current book and section number around and display these.</p>
In order to get this done, we use an extension of StatefulWidget</a> named MyHomePage</code> which carries the state. The state itself contains the elements of book number, section number and the text to be shown on page. In addition, there are methods for manipulating the state and showing it. </p>
For setting the state, we make use of all the functionality we built in meditations.dart</code> to search for the next (or previous, first, last) section and change the state accordingly. Each of these state setting methods are inside a call to setState</code> which makes the Flutter framework be aware of the state changes and, if required, it can redraw the relevant widgets to show these changes.</p>
The methods for showing the state are reasonably straightforward. we mostly use these to translate the internally 0-indexed book- and section numbers into 1 indexed ones to match the actual numbers used in Meditations (albeit not with Roman Numerals though that would be a nice future feature for the app).</p>
The final part is making sure there are widgets for showing the state and interacting with the state. As one can see in the animation of the screen, as well is in the test, we have a section showing the current section and page number, a text area that scrolls showing the current section and a bottom navigation bar to change the current section. </p>
The code for all of this, when put together, can be seen below:</p>
class </span>MyHomePage </span>extends </span>StatefulWidget</span> {
</span>  </span>// The homepage of the app, which also creates and holds the state.
</span>  </span>MyHomePage</span>({</span>Key</span> key, </span>this</span>.title}) : </span>super</span>(key: key);
</span>
</span>  </span>final </span>String</span> title;
</span>
</span>  </span>@override
</span>  </span>_MyHomePageState </span>createState</span>() => </span>_MyHomePageState</span>();
</span>}
</span>
</span>class </span>_MyHomePageState </span>extends </span>State</span><</span>MyHomePage</span>> {
</span>  </span>// The variables used to describe the state.
</span>  </span>int</span> _bookNr = </span>0</span>;
</span>  </span>int</span> _sectionNr = </span>0</span>;
</span>  </span>String</span> _text = (</span>Meditations</span>.</span>getFirst</span>() </span>as </span>BookSectionText</span>).text;
</span>
</span>  </span>// Functions to show the state variables on the screen.
</span>  </span>String </span>getShowText</span>() {
</span>    </span>return</span> _text;
</span>  }
</span>
</span>  </span>int </span>getShowBookNr</span>() {
</span>    </span>// The screen should book and section numbers starting with 1 instead of 0.
</span>    </span>return</span> _bookNr + </span>1</span>;
</span>  }
</span>
</span>  </span>int </span>getShowSectionNr</span>() {
</span>    </span>return</span> _sectionNr + </span>1</span>;
</span>  }
</span>
</span>  </span>// Functions to navigate the text of Meditations and update the state based on UI interaction.
</span>  </span>void </span>_first</span>() {
</span>    </span>// The call to setState makes the Flutter framework aware of the state changes.
</span>    </span>// It will ensure that the build method below will be rerun.
</span>    </span>setState</span>(() {
</span>      </span>BookSectionSearchResult</span> result = </span>Meditations</span>.</span>getFirst</span>();
</span>      </span>if</span> (result is </span>BookSectionText</span>) {
</span>        _text = result.text;
</span>        _bookNr = result.bookNr;
</span>        _sectionNr = result.sectionNr;
</span>      }
</span>    });
</span>  }
</span>
</span>  </span>void </span>_previous</span>() {
</span>    </span>setState</span>(() {
</span>      </span>BookSectionSearchResult</span> result =
</span>          </span>Meditations</span>.</span>getPreviousSection</span>(_bookNr, _sectionNr);
</span>      </span>if</span> (result is </span>BookSectionText</span>) {
</span>        _text = result.text;
</span>        _bookNr = result.bookNr;
</span>        _sectionNr = result.sectionNr;
</span>      }
</span>    });
</span>  }
</span>
</span>  </span>void </span>_next</span>() {
</span>    </span>setState</span>(() {
</span>      </span>BookSectionSearchResult</span> result =
</span>          </span>Meditations</span>.</span>getNextSection</span>(_bookNr, _sectionNr);
</span>      </span>if</span> (result is </span>BookSectionText</span>) {
</span>        _text = result.text;
</span>        _bookNr = result.bookNr;
</span>        _sectionNr = result.sectionNr;
</span>      }
</span>    });
</span>  }
</span>
</span>  </span>void </span>_last</span>() {
</span>    </span>setState</span>(() {
</span>      </span>BookSectionSearchResult</span> result = </span>Meditations</span>.</span>getLast</span>();
</span>      </span>if</span> (result is </span>BookSectionText</span>) {
</span>        _text = result.text;
</span>        _bookNr = result.bookNr;
</span>        _sectionNr = result.sectionNr;
</span>      }
</span>    });
</span>  }
</span>
</span>  </span>// Handles the action of the bottom navigation bar on screen.
</span>  </span>void </span>_onBottomNavTapped</span>(</span>int</span> index) {
</span>    </span>setState</span>(() {
</span>      </span>switch</span> (index) {
</span>        </span>case </span>0</span>:
</span>          {
</span>            </span>_first</span>();
</span>          }
</span>          </span>break</span>;
</span>
</span>        </span>case </span>1</span>:
</span>          {
</span>            </span>_previous</span>();
</span>          }
</span>          </span>break</span>;
</span>
</span>        </span>case </span>2</span>:
</span>          {
</span>            </span>_next</span>();
</span>          }
</span>          </span>break</span>;
</span>
</span>        </span>case </span>3</span>:
</span>          {
</span>            </span>_last</span>();
</span>          }
</span>          </span>break</span>;
</span>
</span>        </span>default</span>:
</span>          {
</span>            </span>_first</span>();
</span>          }
</span>          </span>break</span>;
</span>      }
</span>    });
</span>  }
</span>
</span>  </span>@override
</span>  </span>Widget </span>build</span>(</span>BuildContext</span> context) {
</span>    </span>// This method is rerun every time setState is called.
</span>    </span>// It contains a nested widget with a app bar on top, a scrollable text area in the middle, and a navigation bar at the bottom.
</span>    </span>return </span>Scaffold</span>(
</span>      appBar: </span>AppBar</span>(
</span>        title: </span>Text</span>(
</span>            </span>'${</span>widget.title</span>} Book ${</span>getShowBookNr()</span>} Section ${</span>getShowSectionNr()</span>}'</span>),
</span>      ),
</span>      body: </span>Center</span>(
</span>        child: </span>Column</span>(
</span>          children: <</span>Widget</span>>[
</span>            </span>new </span>Expanded</span>(
</span>                flex: </span>1</span>,
</span>                child: </span>SingleChildScrollView</span>(
</span>                    child: </span>Container</span>(
</span>                      padding: </span>const </span>EdgeInsets</span>.</span>all</span>(</span>8.0</span>),
</span>                      child: </span>Text</span>(</span>getShowText</span>())))),
</span>          ],
</span>        ),
</span>      ),
</span>      bottomNavigationBar: </span>BottomNavigationBar</span>(
</span>        type: </span>BottomNavigationBarType</span>.fixed,
</span>        items: </span>const </span><</span>BottomNavigationBarItem</span>>[
</span>          </span>BottomNavigationBarItem</span>(
</span>            icon: </span>Icon</span>(</span>Icons</span>.first_page),
</span>            label: </span>'First'</span>,
</span>          ),
</span>          </span>BottomNavigationBarItem</span>(
</span>            icon: </span>Icon</span>(</span>Icons</span>.navigate_before),
</span>            label: </span>'Previous'</span>,
</span>          ),
</span>          </span>BottomNavigationBarItem</span>(
</span>            icon: </span>Icon</span>(</span>Icons</span>.navigate_next),
</span>            label: </span>'Next'</span>,
</span>          ),
</span>          </span>BottomNavigationBarItem</span>(
</span>            icon: </span>Icon</span>(</span>Icons</span>.last_page),
</span>            label: </span>'Last'</span>,
</span>          ),
</span>        ],
</span>        onTap: _onBottomNavTapped,
</span>      ),
</span>    );
</span>  }
</span>}
</span></code></pre>
This is pretty much the full description of the Meditations app. It shows how a small application can be built up and structured. My experience with the Flutter framework was very pleasant. Most things I wanted to do were pretty straightforward and there is a large amount of documentation for the framework and setup. I like the emphasis on getting tests up and running straight from the box and the large set of pre-made widgets help a lot with getting a workable UI going very quickly. My biggest pain point was the setup of the whole Android tool-chain itself. Although there were some additional features to the Dart language (such as the upcoming, at the time of this writing, Null Safety</a>) the language is easy to pick up and become productive with.</p>
Of course there are tons of possible extensions to the app. Aside from including the full text of Meditations, one can improve upon the navigation, add features such as bookmarking, random section selection and more. Refactoring and extending the app so it could load in other advice/text in similar format is also a possibility. </p>
Hopefully this article and code was clear enough to follow and my thought process came over clearly. The project has definitely increased my interest in using Dart/Flutter in another project and felt like good practice. Hopefully your next project, perhaps made with Flutter, will be similarly enjoyable and enriching to do! </p>


To Curry in League of Legends
2020-07-04T00:00:00+00:00
One of the most well-known and popular online multi-player games is League of Legends</a>. In this game two teams of five players compete against other. Within the game each player controls a champion that they can use to defeat the enemy team by destroying their base. There are many aspects required to winning a game of League of Legends: good in-game strategy, good control of champions but also selecting a good composition of 5 champions, out of more than a hundred, that fits the needs of the players and play-styles.</p>
One of the lesser-known, but nonetheless very innovative, programming languages is Curry</a>. What makes this language so unique that it is a functional logic language. This means that it combines features of functional programming, composing a program through functions, and logic programming, defining the program as predicates of a logic system and searching for answers. </p>
This article aims to give an introduction to the Curry language by creating a small system in it that matches and recommends League Of Legends champions based on player preferences.</p>
If you want to code along with this article, one can use SMAP</a> to evaluate Curry code online, aside from installing a Curry language implementation such as PACKS</a> locally. The code described in this article can be found in a source repository</a> in its entirety as well.</p>
In order to build such a recommendation system, the first thing we need to do is figure out a way to represent both the player preferences and the League of Legends champions in Curry.</p>
To start off, let's define the aspects of a champion in Curry as follows:</p>
module </span>Main </span>where
</span>
</span>data </span>Tag </span>= </span>Fighter </span>| </span>Tank </span>| </span>Marksman </span>| </span>Assassin </span>| </span>Mage </span>| </span>Support
</span>         </span>deriving</span> (</span>Show</span>, </span>Eq</span>)
</span>                                       
</span>data </span>Champion </span>=
</span>     </span>Champion</span> {name :: </span>String</span>,
</span>              </span>attack </span>:: Int</span>,
</span>              </span>defense </span>:: Int</span>,
</span>              </span>magic </span>:: Int</span>,
</span>              </span>difficulty </span>:: Int</span>,
</span>              </span>tags </span>::</span> []</span>Tag</span>}
</span>     </span>deriving</span> (</span>Show</span>, </span>Eq</span>)
</span></code></pre>
Lets just go over the above code step-by-step. </p>
The first line is the module declaration, which is essentially a grouping of related code together. We will define all elements required for our champion recommendation program in this Main module.</p>
Next we define the data types for our representation of Champion Tags and Champions. Curry is a statically typed language, meaning you can define the structure of the data as types, and you can check these types at compile time before the actual program is run. For example, it only allows the following Tags to be used for the Champions: Fighter, Tank, Marksman, Assassin, Mage, Support. Similarly, a Champion is a record of very specific keys and types of values. For example it has a field for name</code> that must be a string of characters, a field for attack</code> that must be an integer number and a field for tags</code> which must be a list of Tags that we previously defined.</p>
These types allow us to define various champions avaiable in League of Legends. For example, the champion Aatrox can be defined as: </p>
aatrox </span>:: Champion
</span>aatrox = </span>Champion</span> {name = "</span>Aatrox</span>",
</span>                       attack = </span>8</span>,
</span>                       defense = </span>4</span>,
</span>                       magic = </span>3</span>,
</span>                       difficulty = </span>4</span>,
</span>                       tags = [</span>Fighter</span>, </span>Tank</span>]
</span>                       }
</span></code></pre>
Note that this is a correct champion as it fulfills every criteria we gave to the type of Champion. If we would define this champion in the wrong way, such as: </p>
aatrox </span>:: Champion
</span>aatrox = </span>Champion</span> {name = "</span>Aatrox</span>",
</span>                       attack = "</span>high</span>",
</span>                       defense = </span>4</span>,
</span>                       magic = </span>3</span>,
</span>                       difficulty = </span>4</span>,
</span>                       tags = [</span>Fighter</span>, </span>Tank</span>]
</span>                       }
</span></code></pre>
the compiler of Curry would give us an error, "Type error in record construction", along with some additional information that the attack field has a value that does not match the type.</p>
In this definition we explicitly denoted that Aatrox is of the type champion using the line:</p>
aatrox </span>:: Champion
</span></code></pre>
This is not a strict requirement. Curry can infer the type simply from the constructor of the record Champion</code>. Curry can perform type inference even in many other more complex cases, but for clarity we will give the types of records and functions whenever we can.</p>
The final part of this definition, is the portion:</p>
     </span>deriving</span> (</span>Show</span>, </span>Eq</span>)
</span></code></pre>
Without going into too much detail, these makes instances of Champions, and Tags, easy to compare and print as a string.</p>


    
    
    
        
        Aatrox, a champion in League of Legends Copyright Riot Games</h4>
        
        
    </figcaption>
</figure>
The above code should be very familiar to Haskell</a> programmers! The syntax, the type system and many other aspects of Curry are heavily influenced by Haskell. Even the naming of the two languages have the same origin: they are both named after the logician Haskell Curry</a>.</p>
Now that we got the basics out of the way let us define a lot more champions than just Aatrox. </p>
aatrox </span>:: Champion
</span>aatrox = </span>Champion</span> {name = "</span>Aatrox</span>",
</span>                       attack = </span>8</span>,
</span>                       defense = </span>4</span>,
</span>                       magic = </span>3</span>,
</span>                       difficulty = </span>4</span>,
</span>                       tags = [</span>Fighter</span>, </span>Tank</span>]
</span>                       }
</span>darius </span>:: Champion
</span>darius = </span>Champion</span> {name = "</span>Darius</span>",
</span>                   attack = </span>9</span>,
</span>                   defense = </span>5</span>,
</span>                   magic = </span>1</span>,
</span>                   difficulty = </span>2</span>,
</span>                   tags = [</span>Fighter</span>, </span>Tank</span>]
</span>                  }
</span>
</span>
</span>fiora </span>:: Champion
</span>fiora = </span>Champion</span> {name = "</span>Fiora</span>",
</span>                  attack = </span>10</span>,
</span>                  defense = </span>4</span>,
</span>                  magic = </span>2</span>,
</span>                  difficulty = </span>3</span>,
</span>                  tags = [</span>Fighter</span>, </span>Assassin</span>]
</span>                 }
</span>
</span>gnar </span>:: Champion
</span>gnar = </span>Champion</span> {name = "</span>Gnar</span>",
</span>                 attack = </span>6</span>,
</span>                 defense = </span>5</span>,
</span>                 magic = </span>5</span>,
</span>                 difficulty = </span>8</span>,
</span>                 tags = [</span>Fighter</span>, </span>Tank</span>]
</span>                 }
</span>
</span>irelia </span>:: Champion
</span>irelia = </span>Champion</span> {name = "</span>Irelia</span>",
</span>                  attack = </span>7</span>,
</span>                  defense = </span>4</span>,
</span>                  magic = </span>5</span>,
</span>                  difficulty = </span>5</span>,
</span>                  tags = [</span>Fighter</span>, </span>Assassin</span>]
</span>                  }
</span>
</span>karma </span>:: Champion
</span>karma = </span>Champion</span> {name = "</span>Karma</span>",
</span>                  attack = </span>1</span>,
</span>                  defense = </span>7</span>,
</span>                  magic = </span>8</span>,
</span>                  difficulty = </span>5</span>,
</span>                  tags = [</span>Mage</span>, </span>Support</span>]
</span>                  }
</span>
</span>maokai </span>:: Champion
</span>maokai = </span>Champion</span> {name = "</span>Maokai</span>",
</span>                   attack = </span>3</span>,
</span>                   defense = </span>8</span>,
</span>                   magic = </span>6</span>,
</span>                   difficulty = </span>3</span>,
</span>                   tags = [</span>Tank</span>, </span>Mage</span>]
</span>                  }
</span>
</span>neeko </span>:: Champion
</span>neeko = </span>Champion</span> {name = "</span>Neeko</span>",
</span>                  attack = </span>1</span>,
</span>                  defense = </span>1</span>,
</span>                  magic = </span>9</span>,
</span>                  difficulty = </span>5</span>,
</span>                  tags = [</span>Mage</span>, </span>Support</span>]
</span>                  }
</span>
</span>sylas </span>:: Champion
</span>sylas = </span>Champion</span> {name = "</span>Sylas</span>",
</span>                  attack = </span>3</span>,
</span>                  defense = </span>4</span>,
</span>                  magic = </span>8</span>,
</span>                  difficulty = </span>5</span>,
</span>                  tags = [</span>Mage</span>, </span>Assassin</span>]
</span>                  }
</span>
</span>vayne </span>:: Champion 
</span>vayne = </span>Champion</span> {name = "</span>Vayne</span>",
</span>                  attack = </span>10</span>,
</span>                  defense = </span>1</span>,
</span>                  magic = </span>1</span>,
</span>                  difficulty = </span>8</span>,
</span>                  tags = [</span>Marksman</span>, </span>Assassin</span>]
</span>                  }
</span>
</span>champions </span>::</span> [</span>Champion</span>]
</span>champions = [aatrox,
</span>             darius,
</span>             fiora,
</span>             gnar,
</span>             irelia,
</span>             karma,
</span>             maokai,
</span>             neeko,
</span>             sylas,
</span>             vayne
</span>            ]
</span></code></pre>
The above code defines a total of 10 different Champions that exist in League of Legends, whose data values are derived from Riot games Data Dragon API</a>. For easy access, we define a list of champions as the aptly named champions</code>. Note that in Curry a list is denoted by elements between []</code>. The type for a list of elements is denoted by using these bracket in front of the type, such as []Tag</code>, which denotes the type of a list of Tags.</p>
Now that we have the champion definitions out of the way, we also want to define player preferences that we aim to match champions against. Here our preferences consists of a summoner name, identifying the player, minimum requirements for the champion's attack, defense and magic values, a maximum difficulty rating, as well as tags on type of champions the player prefers.</p>
data </span>Preference </span>=
</span>    </span>Preference</span> {summonerName :: </span>String</span>,
</span>                </span>minAttack </span>:: Int</span>,
</span>                </span>minDefense </span>:: Int</span>,
</span>                </span>minMagic </span>:: Int</span>,
</span>                </span>maxDifficulty </span>:: Int</span>,
</span>                </span>prefTags </span>::</span> []</span>Tag</span>}
</span>     </span>deriving</span> (</span>Show</span>, </span>Eq</span>)
</span></code></pre>
Using this Preference type we define two player preferences, one for Alice and one for Bob, as well as a collection for both of them.</p>
alicePref </span>:: Preference
</span>alicePref = </span>Preference</span> {  summonerName = "</span>Alice</span>",
</span>                          minAttack = </span>2</span>,
</span>                          minDefense = </span>1</span>,
</span>                          minMagic = </span>4</span>,
</span>                          maxDifficulty = </span>3</span>,
</span>                          prefTags = [</span>Mage</span>]
</span>                         }
</span>
</span>bobPref </span>:: Preference
</span>bobPref = </span>Preference</span> {  summonerName = "</span>Bob</span>",
</span>                        minAttack = </span>2</span>,
</span>                        minDefense = </span>5</span>,
</span>                        minMagic = </span>2</span>,
</span>                        maxDifficulty = </span>1</span>,
</span>                        prefTags = [</span>Tank</span>, </span>Support</span>]}
</span>
</span>preferences </span>::</span> [</span>Preference</span>]
</span>preferences = [ alicePref,
</span>                bobPref
</span>              ]
</span></code></pre>
We have now defined both the champions and the player preferences and we can move towards creating the functionality to match them together. But first a bit of introduction, or a refresher depending on the readers background, to functional programming. </p>
As mentioned at the start of this article, Curry is a language that supports both functional and logical programming. Functional programming allows us to express the program we want to write as a composition of functions.</p>
To use an example, suppose we want to write a program that given a list of champions, returns only those champion that have an attack of more than 5. In order to do this we can first write a function that given a champion returns true if it has an attack more than 5 and returns false if this is not the case. Then we can use this function as filter inside another function, that only returns members of a list of champions that have more than 5 attack.</p>
To put this together, such a function would look as follows:</p>
highAttackChamps </span>::</span> [</span>Champion</span>] </span>-></span> [</span>Champion</span>]
</span>highAttackChamps champList = filter (\ champion -> attack champion ></span>5</span>) champList
</span></code></pre>
There are a couple of things to note here. </p>
First is that the function that "returns true only if the attack of a champion is greater than 5" has no name. Such anonymous functions, also known as lambda expressions, can be defined using a form starting with a backslash (\) followed by the parameter names of the function, followed by an arrow (->) and the expressions that this function should execute. In this case this expression first gets the attack of the champion and compares it, to give the right result. Anonymous functions are very convenient for expressions that we do not aim to reuse somewhere else.</p>
The second thing to note is the function composition that allows the filtering. The filter</code> function is a predefined function that is imported by default, as a part of a group of functions such as these called a Prelude. As mentioned before this takes a function as a parameter that returns a Bool (True or False), as well as a list, and returns only those members of the list for which the function returned true. Using a function as an argument makes the filter</code> function very versatile. If we, for example wanted to filter based on champion defense, we could have just replaced function used as an argument. Such a composition using functions is one of the core elements of what makes functional programming so effective.</p>
The final thing to note is how the highAttackChamps</code> is given a type. The type declaration highAttackChamps :: [Champion] -> [Champion]</code> shows that this function takes a list of champions and returns a list of champions, where the arrow -></code> functions as a separator between the type of the parameter(s) and/or the type of returned value.</p>
Now that we got over the basics of functional programming lets start to put this knowledge to good use by writing a number of functions that helps us match the Preferences with the Champions. These functions are designed to match a single criteria within a Preference with the value of a Champion. If this match succeeds that the function returns True, if it fails it returns a False. We have a function to match based on attack, defense, magic and difficulty. </p>
matchChampionAttack </span>:: Preference -> Champion -> Bool
</span>matchChampionAttack pref champ = (minAttack pref) <= (attack champ) 
</span>
</span>matchChampionDefense </span>:: Preference -> Champion -> Bool
</span>matchChampionDefense pref champ = (minDefense pref) <= (defense champ) 
</span>
</span>matchChampionMagic </span>:: Preference -> Champion -> Bool
</span>matchChampionMagic pref champ = (minMagic pref) <= (magic champ) 
</span>
</span>matchChampionDifficulty </span>:: Preference -> Champion -> Bool
</span>matchChampionDifficulty pref champ = (maxDifficulty pref) >= (difficulty champ) 
</span></code></pre>
Note that the type definition now have three elements separated by an arrow (->), as these functions have two parameters, a Preference and a Champion, and return a Bool (the type for True or False).</p>
For matching the preferences, we are going to explore some of the logic programming features of Curry. In logic programming the program can be defined as a set of facts, rules and (logic) variables, where the underlying logic system can search for a suitable solution given the criteria.</p>
Give given an example, suppose we are looking for Tag that is both part of the preferred tags of the Preferences as well as part of the tags of the Champions. We can state this problem exactly like this in Curry as follows:</p>
matchChampionTag </span>:: Preference -> Champion -> Bool
</span>matchChampionTag pref champ = (elem tag (prefTags pref) && elem tag (tags champ)) =:= </span>True </span>where</span> tag free
</span></code></pre>
To go over the elements in this code (elem tag (prefTags pref) && elem tag (tags champ))</code> express that tag</code> has to be an element of both the prefTags</code> of the a given preference and tags</code> of the given champion. Unlike with our purely functional code above, we do not give the value of tag</code> as a parameter. Instead we say that this variable is a free variable and let Curry find the values of Tag which makes the above constraint hold True  (i.e. the =:= True where tag free</code> portion of the function).</p>
Now that we have all the functions for matching individual elements (i.e.: attack, defense, magic, difficulty and tags) of champions to profiles, we can create combined functions such as matchChampionAny</code> that holds true if any of the elements match and matchChampionAll</code> for which all elements need to match.</p>
matchChampionAny </span>:: Preference -> Champion -> Bool
</span>matchChampionAny pref champ =  matchChampionAttack pref champ ||
</span>                               matchChampionDefense pref champ ||
</span>                               matchChampionMagic pref champ ||
</span>                               matchChampionDifficulty pref champ ||
</span>                               matchChampionTag pref champ
</span>
</span>
</span>matchChampionAll </span>:: Preference -> Champion -> Bool
</span>matchChampionAll pref champ =  matchChampionAttack pref champ &&
</span>                                 matchChampionDefense pref champ &&
</span>                                 matchChampionMagic pref champ &&
</span>                                 matchChampionDifficulty pref champ &&
</span>                                 matchChampionTag pref champ
</span></code></pre>
One of the coolest features of this language, which we can see in the above definition, is the way logic programming elements can mix with functional elements. In both matchChampionAny</code> and matchChampionAll</code>  we use functions that make use of functional programming, as well as logic programming, seamlessly. </p>
Once we got our functions for matching we can use these to match, we finally have all the ingredients to create recommendations. We are going to create two recommendation functions: one for matching based on matchChampionAny</code> which is a "weak" criteria as any there are lot of opportunities to match and one based on matchChampionAll</code> which is a lot "stronger" as all stats of a champion has to match with the preferences.</p>
The functions for these is given as follows: </p>
recommendWeak :: Preference -> [Champion] -> [String]
</span>recommendWeak preference champList = map (\ champion -> name champion) (filter (\ champion -> matchChampionAny preference champion ) champList)
</span>
</span>recommendStrong :: Preference -> [Champion] -> [String]
</span>recommendStrong preference champList = map (\champion -> name champion) (filter (\ champion -> matchChampionAll preference champion ) champList)
</span></code></pre>
The constructs in here should look familiar with one exception: the use of map (\champion -> name champion)</code>. The map</code> function is similar to filter</code> in takes a function for a parameter as well as a list. Instead of filtering, in instead applies that function to all members of that list. The anonymous function used takes a champion and returns only the name value of that champion. This allows for the matching function to return not a list of champions, but a list of strings i.e.: [String]</code> that are the names of the champions. This makes the return value much less verbose, as we can identify champions by name. </p>
Finally, in order to show off the code detailed here with, we put together some examples of using the all the functions we defined. We then print the answers out, line by line for each of these examples. The main function, with the return type IO (), allows us to not to return a specific value but print the values out, to the REPL or the command-line, depending on how we use our code.</p>
example1 </span>::</span> [</span>Champion</span>]
</span>example1 = highAttackChamps champions
</span>
</span>example2 </span>:: Bool
</span>example2 = matchChampionAttack alicePref aatrox
</span>
</span>example3 </span>:: Bool
</span>example3 = matchChampionDifficulty bobPref vayne 
</span>
</span>example4 </span>:: Bool
</span>example4 =  matchChampionTag alicePref karma
</span>
</span>example5 </span>:: Bool
</span>example5 =  matchChampionAny alicePref neeko
</span>
</span>example6 </span>:: Bool
</span>example6 =  matchChampionAll bobPref gnar
</span>
</span>example7 </span>::</span> [</span>String</span>]
</span>example7 =  recommendWeak alicePref champions
</span>
</span>example8 </span>::</span> [</span>String</span>]
</span>example8 =  recommendWeak bobPref champions
</span>
</span>example9 </span>::</span> [</span>String</span>]
</span>example9 =  recommendStrong alicePref champions
</span>
</span>example10 </span>::</span> [</span>String</span>]
</span>example10 =  recommendStrong bobPref champions
</span>
</span>main </span>:: IO </span>()
</span>main = putStrLn ("</span>Example 1, champions with more than 5 attack:</span>" ++ "</span>\n</span>" ++
</span>                 (show example1) ++ "</span>\n</span>" ++
</span>                 "</span>Example 2, matching Aatrox with Alice's preferences based on attack:</span>" ++ "</span>\n</span>" ++
</span>                 (show example2) ++ "</span>\n</span>" ++
</span>                 "</span>Example 3, matching Vayne with Bob's preferences based on difficulty:</span>" ++ "</span>\n</span>" ++
</span>                 (show example3) ++ "</span>\n</span>" ++
</span>                 "</span>Example 4, matching Karma with Alice's preferences based on tag(s):</span>" ++ "</span>\n</span>" ++
</span>                 (show example4) ++ "</span>\n</span>" ++
</span>                 "</span>Example 5, matching Neeko with Alice's preferences based on any matching criteria:</span>" ++ "</span>\n</span>" ++
</span>                 (show example5) ++ "</span>\n</span>" ++
</span>                 "</span>Example 6, matching Gnar with Bob's preferences based on all matching criteria:</span>" ++ "</span>\n</span>" ++
</span>                 (show example6) ++ "</span>\n</span>" ++
</span>                 "</span>Example 7, recommendation based on a weak criteria for Alice:</span>" ++ "</span>\n</span>" ++
</span>                 (show example7) ++ "</span>\n</span>" ++
</span>                 "</span>Example 8, recommendation based on a weak criteria for Bob:</span>" ++ "</span>\n</span>" ++
</span>                 (show example8) ++ "</span>\n</span>" ++
</span>                 "</span>Example 9, recommendation based on a strong criteria for Alice:</span>" ++ "</span>\n</span>" ++
</span>                 (show example9) ++ "</span>\n</span>" ++
</span>                 "</span>Example 10, recommendation based on a strong criteria for Bob:</span>" ++ "</span>\n</span>" ++
</span>                 (show example10))
</span></code></pre>
This results in the following being printed out:</p>
Example 1</span>, champions with more than </span>5</span> attack:
</span>[</span>Champion</span> {name = "</span>Aatrox</span>", attack = </span>8</span>, defense = </span>4</span>, magic = </span>3</span>, difficulty = </span>4</span>, tags = [</span>Fighter</span>,</span>Tank</span>]},</span>Champion</span> {name = "</span>Darius</span>", attack = </span>9</span>, defense = </span>5</span>, magic = </span>1</span>, difficulty = </span>2</span>, tags = [</span>Fighter</span>,</span>Tank</span>]},</span>Champion</span> {name = "</span>Fiora</span>", attack = </span>10</span>, defense = </span>4</span>, magic = </span>2</span>, difficulty = </span>3</span>, tags = [</span>Fighter</span>,</span>Assassin</span>]},</span>Champion</span> {name = "</span>Gnar</span>", attack = </span>6</span>, defense = </span>5</span>, magic = </span>5</span>, difficulty = </span>8</span>, tags = [</span>Fighter</span>,</span>Tank</span>]},</span>Champion</span> {name = "</span>Irelia</span>", attack = </span>7</span>, defense = </span>4</span>, magic = </span>5</span>, difficulty = </span>5</span>, tags = [</span>Fighter</span>,</span>Assassin</span>]},</span>Champion</span> {name = "</span>Vayne</span>", attack = </span>10</span>, defense = </span>1</span>, magic = </span>1</span>, difficulty = </span>8</span>, tags = [</span>Marksman</span>,</span>Assassin</span>]}]
</span>Example 2</span>, matching </span>Aatrox</span> with </span>Alice</span>'s preferences based on attack:
</span>True
</span>Example 3</span>, matching </span>Vayne</span> with </span>Bob</span>'s preferences based on difficulty:
</span>False
</span>Example 4</span>, matching </span>Karma</span> with </span>Alice</span>'s preferences based on tag(s):
</span>True
</span>Example 5</span>, matching </span>Neeko</span> with </span>Alice</span>'s preferences based on any matching criteria:
</span>True
</span>Example 6</span>, matching </span>Gnar</span> with </span>Bob</span>'s preferences based on all matching criteria:
</span>False
</span>Example 7</span>, recommendation based on a weak criteria for </span>Alice</span>:
</span>["</span>Aatrox</span>","</span>Darius</span>","</span>Fiora</span>","</span>Gnar</span>","</span>Irelia</span>","</span>Karma</span>","</span>Maokai</span>","</span>Neeko</span>","</span>Sylas</span>","</span>Vayne</span>"]
</span>Example 8</span>, recommendation based on a weak criteria for </span>Bob</span>:
</span>["</span>Aatrox</span>","</span>Darius</span>","</span>Fiora</span>","</span>Gnar</span>","</span>Irelia</span>","</span>Karma</span>","</span>Maokai</span>","</span>Neeko</span>","</span>Sylas</span>","</span>Vayne</span>"]
</span>Example 9</span>, recommendation based on a strong criteria for </span>Alice</span>:
</span>["</span>Maokai</span>"]
</span>Example 10</span>, recommendation based on a strong criteria for </span>Bob</span>:
</span>[]
</span></code></pre>
Hopefully the result nothing is too surprising and it matches all expectations. The result also verifies some of our assumptions: the weak criteria matches everything in this case, but a strong criteria has some nice results: Maokai is well suited for Alice, and Bob probably needs to widen his criteria, given the above 10 champions. </p>
There are many ways we could proceed from this point on. As one could see from the examples the recommendation based on a weak criteria is quite weak, as anything matches, while a strong criteria can be too restrictive at times. Coming up with the different types of matching functions, could be a next step, if one wants to expand upon this small application. With multiple matching functions one can also write a function that takes a matching function as a parameter in order to avoid writing duplicate code for each recommendation function. </p>
Hopefully this article has given you a bit of an intro to the Curry language and why it is so interesting. There exists a number of options for using functional and logic programming separately but this is one of the few languages that aims to combine both. If you wish to explore either paradigms further I wish you the best of luck to curry on with your endeavors. </p>


CybOrg Mode
2020-06-01T00:00:00+00:00
Human beings enhanced by machine parts, often called cyborgs</a>, are a staple of the science-fiction and comic books. The, very aptly named, hero Cyborg</a>, is a good example of this. Due to advanced mechanical modifications he not only has greatly increased strength and the ability to fly, but he is able to interface directly with computer systems to organize and plan ahead. These skills make him an important member of many superhero teams such as the Justice League</a> and Teen Titans</a>.</p>
Thankfully for us, even without any of the cybernetic implants that Cyborg has, there exists some great tools to help organize, document and plan ahead. One of such tools is called Org-mode</a>. Org-mode, also denoted as Org mode or even more simply as Org, is a system for organizing notes, plans, authoring documents and many other tasks. One of its main benefits is that although specialized tools exist to help create and edit Org documents, all Org documents are plain text</a> files. This means that the format is very portable and can be used and edited on any platform which can edit text files. In this article we give a brief introduction to Org mode with an example based on the hero Cyborg.</p>
Org documents can be outlined with a tree structure using headlines. Suppose that we want to describe various information about Cyborg, such as his biography, his powers, his relationships and his schedule. For each of these we can use an asterisk (*</code>) symbol followed by some text to create a headline in the document as follows:</p>
* Biography
</span>* Powers
</span>* Relationships
</span>* Cyborg's Schedule
</span></code></pre>
We can add nested headlines by increasing the number of asterisks in front of a section name. For example, if we want to create two headlines under relationships detailing Cyborgs team affiliations, as well as his villains, we can do that:</p>
* Biography
</span>* Powers
</span>* Relationships
</span>** Team Affiliations
</span>** Villains
</span>* Cyborg's Schedule
</span></code></pre>
Of course while headlines can help us structure a document, text is an integral part of it. A simple biography can be shown as one would expect: </p>
* Biography
</span>Vic Stone was a high school athlete when a tragic accident nearly killed him and destroyed most of this body. His father, Dr. Silas Stone, used his scientific knowledge and advanced technology to save his life by augmenting him with machine parts.
</span>Living now as a cyborg, a being with both organic and machine parts, he is now has super strength, increased durability and a control over advanced technology and weaponry.
</span>Although he continues to struggle with his newfound status as both man and machine, he uses his newfound powers as a superhero.
</span>* Powers
</span>* Relationships
</span>** Team Affiliations
</span>** Villains
</span>* Cyborg's Schedule
</span></code></pre>
We can also create lists in the document. For example we can give an unordered list of Cyborg's powers such as super strength and instant weaponary by prefacing each item on a new line by a hyphen (-</code>): </p>
* Biography
</span>Vic Stone was a high school athlete when a tragic accident nearly killed him and destroyed most of this body. His father, Dr. Silas Stone, used his scientific
</span>knowledge and advanced technology to save his life by augmenting him with machine parts.
</span>Living now as a cyborg, a being with both organic and machine parts, he is now has super strength, increased durability and a control over advanced technology and weaponry.
</span>Although he continues to struggle with his newfound status as both man and machine, he uses his newfound powers as a superhero.
</span>* Powers
</span>- super strength
</span>- advanced technology,
</span>- instant weaponry
</span>- genius-level intellect
</span>- control over technology
</span>- computer hacking
</span>- durability
</span>- teleportation
</span>* Relationships
</span>** Team Affiliations
</span>** Villains
</span>* Cyborg's Schedule
</span></code></pre>
Note that the document is now getting larger and the parts of what we want to focus on to show here, the powers, can get lost in the other information. One of the strengths of Org-mode that even though it can be viewed and edited as plain text, there exists quite a bit of tooling to help navigate, edit and make use of the described structures. The editor emacs</a> and its various flavours such as spacemacs</a> that I personally use have excellent features for these tasks. However other editors such as Visual Studio Code</a> have plugins</a> to support Org-mode as well.</p>
One of the basic features to help such editors provide is to fold headlines under which we do not want to see all the content in detail and unfold it for those that we do. In the following animation we show how this can be done on a portion of a more extended version of the Cyborg document: </p>


    
    
    
        
        Folding and unfolding headlines in Org document using Spacemacs</h4>
        
        
    </figcaption>
</figure>

The above image shows how the spacemacs</a> editor handles Org documents. By simply pressing tab on headlines we can fold (i.e. not show the underlying content) and unfold (i.e. showing the underlying content) them as we please and directing our attention to parts of the document we want to view and edit. It also shows how editor support uses the underlying structure for additional visualizations, such as on the level of headlines, even though the underlying document is still plaintext.</p>
Expanding on our example we can describe the Team Affiliations and the Villains as lists as well. For the list of Team Affiliations we will use an ordered list, which can be done by using a number followed by a dot, e.g.: 1. 2.</code> for the list indicators.  Note for brevity we now only show the new/relevant portions of the full Org document in the examples as opposed to the whole.</p>
* Relationships
</span>** Team Affiliations
</span>1. Justice League
</span>2. Teen Titans
</span>3. Doom Patrol
</span>** Villains
</span>- Lex Luthor
</span>- Deathstroke
</span></code></pre>
Another interesting feature in Org documents are TODO items. Any headline can be made a TODO in them by prefacing it with TODO. For example in a headline of Cyborg's schedule we can put two TODO items for things Cyborg wants to do at some point: attend a Justice League Meeting and Upgrading Equipment. </p>
There are various ways such TODO items could be used similarly to folding or unfolding headlines.  In emacs/spacemacs such actions, and many others relating to various aspects of an Org document, can be done by commands. For example the command org-show-todo-tree</code> can take ensure that all the headlines are folded as much as possible and only the TODO headlines are unfolded. Similarly the command org-todo</code> cycles between not marking the headline with a TODO, marking it with a TODO or marking it with DONE denoted that the item has been completed. </p>


    
    
    
        
        Toggling using the command the command `org-todo` for headlines in Org document using Spacemacs</h4>
        
        
    </figcaption>
</figure>

The above image shows how the command org-todo</code> can cycle between the TODO states of the Justice League Meeting headline. In addition, it also shows a few other features. </p>
First is the associating a date with the TODO item, in particular when it is scheduled. By toggling the TODO to DONE, our editor uses the current time to mark when the task was finished. This allows for a nice way to create a checklist of the things that have to be done and marking them off.</p>
Another aspect is the use of markup around 'must' and 'Lex Luthor'. Enclosing a phrase with an underscore underlines the phrase while enclosing it between asterisks (*) makes it bold. Markups for italicizing (enclosing it between slashes (/)), code fragments and others also exist. </p>
The full list of TODO items in Cyborg's schedule is as follows.</p>
*** </span>TODO</span> Justice League Meeting
</span>    SCHEDULED: <2024-08-02>
</span>    Something _must_ be done about </span>*Lex Luthor*</span>. He seems to be planning an attack on </span>/Metropolis/</span> with the help of </span>*Deathstroke*</span>. 
</span>*** </span>TODO</span> Upgrade Equipment
</span>    SCHEDULED: <2024-06-01>
</span>    Enhance arm cannon.
</span></code></pre>
One of the most interesting features of Org mode that we will cover in this article is the ability to embed fragment of code in an Org document that can be executed in the right environment. In Cyborg's schedule one of the upcoming items is to enhance his arm cannon. In order to prepare for this, and to illustrate our code embedding, a simple Weapon Damage calculator application was written in Python:</p>
  arm_cannon = {"</span>name</span>": "</span>Arm Cannon</span>", "</span>power</span>": </span>8</span>}
</span>  reinforced_door = {"</span>name</span>": "</span>Reinforced Door</span>", "</span>armor</span>": </span>3</span>}
</span>
</span>  </span>def </span>wdc</span>(</span>weapon</span>, </span>target</span>):
</span>    damage = weapon["</span>power</span>"] - target["</span>armor</span>"]
</span>    </span>if </span>damage<</span>0</span>:
</span>      </span>return </span>0
</span>    </span>else</span>: 
</span>      </span>return </span>damage
</span>
</span>  damage_done = </span>wdc</span>(arm_cannon, reinforced_door)
</span>  </span>print</span>("</span>Using the weapon </span>" + arm_cannon["</span>name</span>"] + "</span> with </span>" + </span>str</span>(arm_cannon["</span>power</span>"]) + "</span> power,</span>")
</span>  </span>print</span>("</span>against the target </span>" + reinforced_door["</span>name</span>"] + "</span> with </span>" + </span>str</span>(reinforced_door["</span>armor</span>"]) + "</span> armor,</span>")
</span>  </span>print</span>("</span>results in </span>" + </span>str</span>(damage_done) + "</span> damage.</span>")
</span></code></pre>
This code aims to use features of Cyborg's arm cannon to calculate damage on the target and to write these features out along with the resulting damage.</p>
We can embed this code fragment by using the property system to give this fragment a name and denote some other attributes. This property system is essentially a key-value system in Org mode that is used for all types information represented with key-value pairs. For example the following two properties describe the title and description of the Org document:</p>
#+</span>title</span>: </span>Cyborg Agenda
</span>#+</span>description</span>: </span>An introduction to org mode by creating an org document for information on the superhero Cyborg.
</span></code></pre>
For the code we want to embed we want set the properties on how we name this fragment, where the source begins and ends, what language it is and what should be done with the results. This looks as follows within the Org document:</p>
* Weapon Damage Calculator
</span>#+</span>NAME</span>: </span>wdc
</span>#+</span>BEGIN_SRC </span>python </span>:results output
</span>  arm_cannon = {"name": "Arm Cannon", "power": 8}
</span>  reinforced_door = {"name": "Reinforced Door", "armor": 3}
</span>
</span>  def wdc(weapon, target):
</span>    damage = weapon["power"] - target["armor"]
</span>    if damage<0:
</span>      return 0
</span>    else: 
</span>      return damage
</span>
</span>  damage_done = wdc(arm_cannon, reinforced_door)
</span>  print("Using the weapon " + arm_cannon["name"] + " with " + str(arm_cannon["power"]) + " power,")
</span>  print("against the target " + reinforced_door["name"] + " with " + str(reinforced_door["armor"]) + " armor,")
</span>  print("results in " + str(damage_done) + " damage.")
</span>#+</span>END_SRC
</span>#+</span>RESULTS</span>: </span>wdc
</span></code></pre>
This embeds the weapon damage calculator as a source fragment named wdc, while also setting up where the results will be show in the document. Now if we execute the command org-babel-execute-src-block</code> we can see the results directly within the document:</p>


    
    
    
        
        Executing the source block to see the results directly within the Org document</h4>
        
        
    </figcaption>
</figure>

That is it for this tutorial, hopefully I have given a glimpse of what Org mode can do, and gives you some idea on how it can enhance your organizing, planning and note taking. Here I have not even touched on many other great features such as authoring documents and converting to other formats, using it as a spreadsheet system and many more. For further resources on this tool take a look at the official Org mode website</a>, as well the excellent video tutorial</a> by GDQuest on using the Spacemacs editor with Org mode. The Org document that was built up can also be found in its entirety at a github repository</a> along with some extra documentation.</p>
Hopefully you have enjoyed the article and if you want to enhance yourself with one of the best note taking and planning tools around give Org mode a shot.</p>


Darmok in core.logic
2020-02-02T00:00:00+00:00
One of the greatest Star Trek episodes is titled Darmok</a> in Star Trek: The Next Generation</a>.  It has the hallmarks of a great Star Trek: TNG episode: a first contact between two civilizations and a dilemma that is not solved by violence but by thinking and understanding.</p>
One of the nicest logic programming languages is miniKanren</a>. This is due to the fact that it is a small, relatively easy to understand Logic Programming (LP)</a> language and has implementations in many programming languages. This later feature allows for logic programming features to be used in many different environments, as a logic programming Domain-Specific language (DSL)</a>. </p>
A popular implementation of miniKanren is the core.logic</a> library of the Clojure</a> programming language. In this article we aim to introduce miniKanren/core.logic by encoding story elements of the Darmok episode of Star Trek: TNG (some spoilers for the episode will follow). </p>
The episode is based around the fact that the Federation and the Tamarian people aim to establish successful first contact with each other. From the Federation, the crew of the starship Enterprise are sent to the planet El-Adrel where a Tamarian ship awaits them. Unfortunately attempts at communication fail from both sides and lead to some dangerous situations. The difficulty of communication arises from the fact that Tamarians communicate exclusively through allegory. This means that it is not enough to just decipher the words and grammar used in the Tamarian language, but the crew of the Enterprise must also understand the myths and historical events to which these allegories refer to. Within the episode multiple allegories are used by the Tamarians, such as with the phrase "Darmok and Jalad at Tanagra", that utterly baffle the crew at first. However due to shared dangers and cooperation by the captain of the Enterprise, Picard, and the captain of the Tamarian ship, Darmok, they start to understand each other. In the end a successful first contact is made. This is captured by the newly coined allegory for first contact in the Tamarian language: "Picard and Dathon at El-Adrel".</p>
In this article we will use core.logic to write a logic program to represent the allegories used in the Darmok episode and to generate templates of the Darmok story through a sequence of allegories. A logic program is a bit different than the programs most people are used to. Instead of giving precise instructions to the computer one instead writes a goal, or a group of goals, that provide some logical restrictions on what one intends to achieve. With these goals the logic programming system can find the right answers. Note that the source for the code used this article can be found here</a> in case you want to experiment along while reading this article.</p>
Let's start off with a small logic program that can translate the Tamarian allegory that represents cooperation, the phrase "Darmok and Jalad at Tanagra". We aim to translate this allegory to an equivalent allegory based on a human myth, as well as the English translation for it: "cooperation". </p>
In order to do this we give the Clojure definition of this goal as a logic program, then explain each element of it and how to use it.</p>
(</span>defn </span>cooperation </span>[tam hum eng]
</span>  (</span>l/conde </span>[(</span>l/== </span>[tam] ["</span>Darmok and Jalad at Tanagra.</span>"])
</span>            (</span>l/== </span>[hum] ["</span>Gilgamesh and Ekidu at Uruk.</span>"])
</span>            (</span>l/== </span>[eng] ["</span>cooperation</span>"])]))
</span></code></pre>
First lets decipher the above definition. For people, unfamiliar with Clojure, the form (defn cooperation [tam hum eng] ... )</code> defines a function named cooperation</code> with the parameters tam hum eng</code>. The part with (l/conde ... )</code> is a shorthand for (core.logic/conde ... )</code>; we will use l</code> as the abbreviation for the core.logic</code> namespace in the rest of this article. This l/conde</code> part functions as a way to connect various goals together. It creates a disjunction (elements separated by OR) of separate vectors of goals which it considers as conjunction (elements separated by AND). For people a bit familiar with boolean logic this a way to write a Disjunctive Normal Form</a>. To give a very simplified example the form (l/conde [A B] [C] )</code> with the goals A, B, C</code> can be seen as a way to find the case where (A 'AND' B) 'OR' C</code> holds. In the previous example l/conde</code> is called on a single vector of elements [(l/== [tam] ["Darmok and Jalad at Tanagra."]) (l/== [hum] ["Gilgamesh and Ekidu at Uruk."]) (l/== [eng] ["cooperation"])]</code>, meaning that this function wants each of the goals: (l/== [tam] ["Darmok and Jalad at Tanagra."])</code> AND (l/== [hum] ["Gilgamesh and Ekidu at Uruk."])</code> AND  (l/== [eng] ["cooperation"])</code> fulfilled. </p>
So now we know that this function takes three parameters and wants to ensure that the three goals all have to be met. But what do the goals themselves mean? They all have a similar structure in that they are using the equality in core.logic, l/==</code>, to unify elements. Unification is a core part of a logic programming system and it is used to constrain elements to the same possible values. For the first example (l/== [tam] ["Darmok and Jalad at Tanagra."])</code>, the unification aims to ensure that the variable tam</code> has the value of the string "Darmok and Jalad at Tanagra."</code>, which can be seen as the Tamarian phrase for cooperation. The other goals do this unification for the human mythology equivalent: "Gilgamesh and Ekidu at Uruk."</code> of this allegory, as well for the English word "cooperation"</code>, for the variables hum</code> and eng</code>, respectively.</p>
We have now given an anatomy of this logic program that does unification on phrases relating to cooperation, but how do we use it? For this we need two things: a set of logic variables and a way to tell the system to run the logic program. The function l\run*</code> does exactly that, which for the given set of parameters tries to find all examples where the goals are fulfilled.</p>
So if we evaluate the following code:</p>
(</span>l/run* </span>[tam hum eng]
</span>        (</span>cooperation</span> tam hum eng))
</span></code></pre>
we get the following:</p>
(["</span>Darmok and Jalad at Tanagra.</span>" "</span>Gilgamesh and Ekidu at Uruk.</span>" "</span>cooperation</span>"])
</span></code></pre>
What l\run*</code> is doing is taking a given list of logic variables and tries to list all the possible values these variables can take. Here it only lists a single possible set of values for the variables tam</code>, hum</code> and eng</code>. This should not be surprising as there is only exactly one way each of the variables tam</code>, hum</code>, and eng</code> can be fulfilled by the "cooperation" goal based on the definition we gave above.  The variable tam</code> gets unified with the string value "Darmok and Jalad at Tanagra."</code>, hum</code> with "Gilgamesh and Ekidu at Uruk."</code> and eng</code> with "cooperation"</code>.</p>
In the previous case we used only 'fresh' logic variables (variables that have no constraints placed upon their possible values yet), but instead we can also use a specific value in our goal instead. In the following example we only have two variables. Instead of a variable for first parameter used in the cooperation goal we give the string `"Darmok and Jalad at Tanagra." as follows:</p>
(</span>l/run* </span>[hum eng]
</span>        (</span>cooperation </span>"</span>Darmok and Jalad at Tanagra.</span>" hum eng))
</span></code></pre>
which gives the result:</p>
(["</span>Gilgamesh and Ekidu at Uruk.</span>" "</span>cooperation</span>"])
</span></code></pre>
There are only two variables listed in the answer, as there are only two variables given for l/run*</code> to check in our initial case. Otherwise the answer is exactly what we would expect as there is only one possible way these variables can be bound in our logic program.</p>
Now lets try a run where there are no possible valid answers, giving the word "challenge"</code> as a parameter:</p>
(</span>l/run* </span>[tam hum eng]
</span>        (</span>cooperation</span> tam hum "</span>challenge</span>"))
</span></code></pre>
This returns an empty list of answers: ()</code>, as there are no ways to unify the word "challenge"</code> inside the goal of cooperation.</p>
Given we got the basics of logic programs covered, lets expand our example into something more complex.</p>
Instead of one single allegory, we now define five of them based on the various allegories used in the Darmok episode. The functions representing these goals are all named after the English word translation: failure</code>, common-enemy</code>, cooperation</code>, successful-cooperation</code> and successful-first-contact</code>. They all follow the same structure as the cooperation</code> allegory we previously examined in detail. In addition we also define a function for representing any allegory, aptly named allegory</code>. This is a goal that can be fulfilled by any of the allegories named above.</p>
(</span>defn </span>failure </span>[tam hum eng]
</span>  (</span>l/conde </span>[(</span>l/== </span>[tam] ["</span>Shaka, when the walls fell.</span>"])
</span>            (</span>l/== </span>[hum] ["</span>Gilgamesh, his plant eaten by a snake.</span>"])
</span>            (</span>l/== </span>[eng] ["</span>failure</span>"])]))
</span>
</span>(</span>defn </span>common-enemy </span>[tam hum eng]
</span>  (</span>l/conde </span>[(</span>l/== </span>[tam] ["</span>Beast at Tanagra.</span>"])
</span>            (</span>l/== </span>[hum] ["</span>Bull of Heaven.</span>"])
</span>            (</span>l/== </span>[eng] ["</span>common-enemy</span>"])]))
</span>
</span>(</span>defn </span>cooperation </span>[tam hum eng]
</span>  (</span>l/conde </span>[(</span>l/== </span>[tam] ["</span>Darmok and Jalad at Tanagra.</span>"])
</span>            (</span>l/== </span>[hum] ["</span>Gilgamesh and Ekidu at Uruk.</span>"])
</span>            (</span>l/== </span>[eng] ["</span>cooperation</span>"])]))
</span>
</span>(</span>defn </span>successful-cooperation </span>[tam hum eng]
</span>  (</span>l/conde </span>[(</span>l/== </span>[tam] ["</span>Darmok and Jalad on the ocean.</span>"])
</span>            (</span>l/== </span>[hum] ["</span>Gilgamesh and Ekidu, after the Bull's defeat.</span>"])
</span>            (</span>l/== </span>[eng] ["</span>successful-cooperation</span>"])]))
</span>
</span>(</span>defn </span>successful-first-contact </span>[tam hum eng]
</span>  (</span>l/conde </span>[(</span>l/== </span>[tam] ["</span>Picard and Dathon at El-Adrel.</span>"])
</span>            (</span>l/== </span>[hum] ["</span>Picard and Dathon at El-Adrel.</span>"])
</span>            (</span>l/== </span>[eng] ["</span>successful-first-contact</span>"])]))
</span>
</span>(</span>defn </span>allegory </span>[tam hum eng]
</span>  (</span>l/conde </span>[(</span>failure</span> tam hum eng)]
</span>           [(</span>common-enemy</span> tam hum eng)]
</span>           [(</span>cooperation</span> tam hum eng)]
</span>           [(</span>successful-cooperation</span> tam hum eng)]
</span>           [(</span>successful-first-contact</span> tam hum eng)]))
</span></code></pre>
There should not be any very surprising elements in this part, but we give two quick observations before continuing. </p>
First, the human equivalent for Tamarian allegories are based on the Gilgamesh story, which is also explicitly mentioned in the episode, that provides a way for captain Picard to connect with the Tamarian captain Dathon. The only exception to this is "Picard and Dathon at El-Adrel."</code> which is an allegory coined at the end of the episode as a term for first contact between cultures. It seems fitting to use this as an allegory from a human perspective as well. </p>
Second, to reiterate how l/conde</code> works with choices, we remark that each of the named allegories in the allegory</code> function are in their own vector (indicated by each goal inside their own []</code> brackets). This indicates that any of failure</code>, common-enemy</code>, cooperation</code>, successful-cooperation</code> or successful-first-contact</code> can fulfill the goal of allegory</code>. </p>
We can test this later notion by running a short logic program for finding all possible allegories:</p>
(</span>l/run* </span>[tam hum eng]
</span>        (</span>allegory</span> tam hum eng))
</span></code></pre>
which returns:</p>
(["</span>Shaka, when the walls fell.</span>"
</span>  "</span>Gilgamesh, his plant eaten by a snake.</span>"
</span>  "</span>failure</span>"]
</span> ["</span>Beast at Tanagra.</span>" "</span>Bull of Heaven.</span>" "</span>common-enemy</span>"]
</span> ["</span>Darmok and Jalad at Tanagra.</span>" "</span>Gilgamesh and Ekidu at Uruk.</span>" "</span>cooperation</span>"]
</span> ["</span>Darmok and Jalad on the ocean.</span>"
</span>  "</span>Gilgamesh and Ekidu, after the Bull's defeat.</span>"
</span>  "</span>successful-cooperation</span>"]
</span> ["</span>Picard and Dathon at El-Adrel.</span>"
</span>  "</span>Picard and Dathon at El-Adrel.</span>"
</span>  "</span>successful-first-contact</span>"])
</span></code></pre>
This shows five possible answers because, as mentioned, any of the above allegories can fulfill the given goal.</p>
By default l/run*</code> will list all possible answers for a given logic program, which is a very powerful feature for exhaustively searching for all the solutions to a given problem. However this list can be large, and even infinite! In such scenarios there is a way to limit the answers to a certain number when searching by using l/run</code> (note the lack of the *</code> character) directly followed by the number of answers we want returned. </p>
For example, the call:</p>
(</span>l/run </span>2 </span>[tam hum eng]
</span>       (</span>allegory</span> tam hum eng))
</span></code></pre>
will return only two possible answers: </p>
(["</span>Shaka, when the walls fell.</span>"
</span>  "</span>Gilgamesh, his plant eaten by a snake.</span>"
</span>  "</span>failure</span>"]
</span> ["</span>Beast at Tanagra.</span>" "</span>Bull of Heaven.</span>" "</span>common-enemy</span>"])
</span></code></pre>
Now as we are getting a bit more familiar with the allegories in this example, we do not want to write out all three versions of each allegory each time. We can do this by defining a new goal allegory-short</code> that succeeds for any phrase that identifies one of the five allegories:</p>
(</span>defn </span>allegory-tam </span>[tam]
</span>  (</span>l/fresh </span>[x y]
</span>           (</span>allegory</span> tam x y)))
</span>
</span>(</span>defn </span>allegory-hum </span>[hum]
</span>  (</span>l/fresh </span>[x y]
</span>           (</span>allegory</span> x hum y)))
</span>
</span>(</span>defn </span>allegory-eng </span>[eng]
</span>  (</span>l/fresh </span>[x y]
</span>           (</span>allegory</span> x y eng)))
</span>
</span>(</span>defn </span>allegory-short </span>[x]
</span>  (</span>l/conde </span>[(</span>allegory-tam</span> x)]
</span>           [(</span>allegory-hum</span> x)]
</span>           [(</span>allegory-eng</span> x)]))
</span></code></pre>
The allegory-short</code> function was defined by writing out the three scenarios by which a phrase could be part of an allegory: it is either the Tamarian allegory, the Human allegory or the English translation. The only new structure we use here from core.logic is l/fresh</code> which lets us introduce new (fresh) logic variables which have no binding as of yet. When using the various forms of l/run</code> the parameters for the function are automatically given as fresh variables, but this function allows us to create them inside other parts of the logic program as well.</p>
Now if we want to list, for example, five possible phrases that form part of an allegory we can call:</p>
(</span>l/run </span>5 </span>[x]
</span>       (</span>allegory-short</span> x))
</span></code></pre>
which will return five of the possible terms that are used as part of allegories.</p>
("</span>Shaka, when the walls fell.</span>"
</span> "</span>Beast at Tanagra.</span>"
</span> "</span>Darmok and Jalad at Tanagra.</span>"
</span> "</span>Gilgamesh, his plant eaten by a snake.</span>"
</span> "</span>failure</span>")
</span></code></pre>
With all the functions for logic programming we built up, lets try our hand at creating a logic program that generates variants of the Darmok story expressed through a sequence of allegories. In this scenario, much like in the episode any phrase, a Tamarian- or Human allegory or their English equivalent, could be used to describe parts of the story. In essence we can represent the story as a list of phrases, for example: ("failure", "Beast at Tanagra." "Darmok and Jalad at Tanagra." "successful-first-contact" ). </p>
We could put many restrictions on the order of the phrases, but for the sake of brevity we just want to ensure that each story starts with a phrase for failure</code> and ends with a phrase for successful-first-contact</code> much like the structure of the actual episode. In addition let's assume our stories are only five phrases long. </p>
We can now define logic program to generate such stories as follows:</p>
(</span>defn </span>failure-any </span>[x]
</span>  (</span>l/fresh </span>[a1 a2 a3]
</span>           (</span>l/conde </span>[(</span>failure</span> x a1 a2)]
</span>                    [(</span>failure</span> a1 x a2)]
</span>                    [(</span>failure</span> a1 a2 x)])))
</span>
</span>(</span>defn </span>successful-first-contact-any </span>[x]
</span>  (</span>l/fresh </span>[a1 a2 a3]
</span>           (</span>l/conde </span>[(</span>successful-first-contact</span> x a1 a2)]
</span>                    [(</span>successful-first-contact</span> a1 x a2)]
</span>                    [(</span>successful-first-contact</span> a1 a2 x)])))
</span>
</span>(</span>defn </span>five-element-story </span>[x1</span>,</span> x2</span>,</span> x3</span>,</span> x4</span>,</span> x5]
</span>  (</span>l/fresh </span>[a1 a2 a3]
</span>           (</span>l/conde </span>[(</span>failure-any</span> x1)
</span>                     (</span>allegory-short</span> x2)
</span>                     (</span>allegory-short</span> x3)
</span>                     (</span>allegory-short</span> x4)
</span>                     (</span>successful-first-contact-any</span> x5)])))
</span></code></pre>
Every construct we used to build these functions should be familiar based on the previous examples. We just needed to define two special versions of our goals for the shorthand version of allegories: one for failure and one for successful first contact.</p>
For example if we want five solutions that fulfill all the criteria for such stories we can call the following code: </p>
(</span>l/run </span>5 </span>[x1 x2 x3 x4 x5]
</span>       (</span>five-element-story</span> x1 x2 x3 x4 x5))
</span></code></pre>
which for example could return:</p>
(["</span>Shaka, when the walls fell.</span>"
</span>  "</span>Shaka, when the walls fell.</span>"
</span>  "</span>Shaka, when the walls fell.</span>"
</span>  "</span>Shaka, when the walls fell.</span>"
</span>  "</span>Picard and Dathon at El-Adrel.</span>"]
</span> ["</span>Shaka, when the walls fell.</span>"
</span>  "</span>Shaka, when the walls fell.</span>"
</span>  "</span>Shaka, when the walls fell.</span>"
</span>  "</span>Shaka, when the walls fell.</span>"
</span>  "</span>Picard and Dathon at El-Adrel.</span>"]
</span> ["</span>Shaka, when the walls fell.</span>"
</span>  "</span>Shaka, when the walls fell.</span>"
</span>  "</span>Shaka, when the walls fell.</span>"
</span>  "</span>Shaka, when the walls fell.</span>"
</span>  "</span>successful-first-contact</span>"]
</span> ["</span>Shaka, when the walls fell.</span>"
</span>  "</span>Shaka, when the walls fell.</span>"
</span>  "</span>Shaka, when the walls fell.</span>"
</span>  "</span>Beast at Tanagra.</span>"
</span>  "</span>Picard and Dathon at El-Adrel.</span>"]
</span> ["</span>Shaka, when the walls fell.</span>"
</span>  "</span>Shaka, when the walls fell.</span>"
</span>  "</span>Shaka, when the walls fell.</span>"
</span>  "</span>Beast at Tanagra.</span>"
</span>  "</span>Picard and Dathon at El-Adrel.</span>"])
</span></code></pre>
As one can see the logic program will exhaustively go through all the possible ways the goals can fulfilled and list them up to limit given as a parameter for l/run</code>. The results are not necessarily unique if there are multiple ways to fulfill the goals. This can be seen in the first and second answers as the phrase "Picard and Dathon at El-Adrel." is both the Tamarian and Human allegory for successful first contact. </p>
The above restrictions allow for a lot of the same allegories used within the story. For this, one can define new restrictions and further fine tune the story generation. For example, one can create restrictions on the number of duplicate phrases used, or could ensure that there is more diversity in the phrase type (Tamarian, Human, English) is used. Declaring new restrictions, combining them with existing ones and using the same mechanism to derive any number of answers is one of the core strengths of a logic programming system such as core.logic.</p>
As a final example to show how logic programming can be embedded into a (regular) program, we create a logic program inside a regular Clojure function that creates n-number of stories of a given length. Take a quick look at the following code:</p>
(</span>defn </span>n-element-story </span>[nr-of-elements stories]
</span>  (</span>let </span>[vars (</span>repeatedly</span> nr-of-elements l/lvar)
</span>        first-var (</span>first</span> vars)
</span>        middle (</span>drop-last </span>(</span>rest</span> vars))
</span>        last-var (</span>last</span> vars)]
</span>    (</span>l/run</span> stories [q]
</span>           (</span>l/conde </span>[(</span>l/==</span> q vars)
</span>                     (</span>l/distincto</span> q)
</span>                     (</span>failure-any</span> first-var)
</span>                     (</span>successful-first-contact-any</span> last-var)
</span>                     (</span>l/everyg</span> allegory-short middle)]))))
</span></code></pre>
Without going too in-depth on every part of this function, it programmatically creates n-number of fresh variables based on the given parameter. It then unifies these with the parameters of a run execution inside the function and returns them. For example, six stories of four elements can be requested by the call:</p>
(</span>n-element-story </span>4 6</span>)
</span></code></pre>
which will result in the stories:</p>
(("</span>Shaka, when the walls fell.</span>"
</span>  "</span>Beast at Tanagra.</span>"
</span>  "</span>Darmok and Jalad at Tanagra.</span>"
</span>  "</span>Picard and Dathon at El-Adrel.</span>")
</span> ("</span>Shaka, when the walls fell.</span>"
</span>  "</span>Beast at Tanagra.</span>"
</span>  "</span>Darmok and Jalad on the ocean.</span>"
</span>  "</span>Picard and Dathon at El-Adrel.</span>")
</span> ("</span>Shaka, when the walls fell.</span>"
</span>  "</span>Beast at Tanagra.</span>"
</span>  "</span>Gilgamesh, his plant eaten by a snake.</span>"
</span>  "</span>Picard and Dathon at El-Adrel.</span>")
</span> ("</span>Shaka, when the walls fell.</span>"
</span>  "</span>Beast at Tanagra.</span>"
</span>  "</span>failure</span>"
</span>  "</span>Picard and Dathon at El-Adrel.</span>")
</span> ("</span>Shaka, when the walls fell.</span>"
</span>  "</span>Beast at Tanagra.</span>"
</span>  "</span>Bull of Heaven.</span>"
</span>  "</span>Picard and Dathon at El-Adrel.</span>")
</span> ("</span>Shaka, when the walls fell.</span>"
</span>  "</span>Darmok and Jalad at Tanagra.</span>"
</span>  "</span>Beast at Tanagra.</span>"
</span>  "</span>Picard and Dathon at El-Adrel.</span>"))
</span></code></pre>
There you have it, a very quick overview of using core.logic for logic programming. If you would like to experiment further the code used in this article is available</a>. Here we only scratched the surface of what is possible in a logic programming environment such as core.logic. Feel free to check it out, or any other miniKanren implementation available in your language</a>.</p>
I hope that, much like the Darmok episode, this article has expanded your horizons on communicating. Logic programming is a very interesting, and often underutilized programming paradigm. The core.logic/miniKanren logic programming language is a great system to get started with it. I hope that this, perhaps first, contact with Logic Programming or core.logic aids you in your future endeavors.</p>


The Probable Mystery Machine
2019-12-06T00:00:00+00:00
Scooby Doo is mystery horror cartoon series in which a group of teenagers named Fred, Daphne, Velma and Shaggy alongside the titular Great Dane named Scooby-Doo, ride around in their van named "The Mystery Machine" solving mysteries. The episodes of the show generally follow a set structure. First their van tends to break down near a place apparently haunted by a ghost or another supernatural creature. They would offer to solve the mystery behind the existence of the monster and start looking for clues. The monster tries to scare them away while they find various pieces of evidence relating to it, all pointing to the fact that the monster is not real. At a certain point the creature starts to chase them until they can trap or otherwise incapacitate it. Finally they figure out that the monster is person in a costume who put the mystery in place to scare people away for some (financial) reason, and who would have gotten away with it "if not for them meddling kids".</p>
While having a van named The Mystery Machine can help solving mysteries, we can turn our computer into a mystery solving machine as well. We can represent possible stories in a Scooby Doo episode using some logical facts (e.g. a each adventure has a monster in it) as well as probabilities (e.g. there is a 40% chance a monster will be a ghost). In particular, we can use a probabilistic logic programming language, namely ProbLog</a> to guide us through a scenario of a Scooby Doo story.</p>
First let's start off with a basic scenario on how a Scooby Doo adventure starts. In general there is usually some sort of an issue why the group must stop during their travels. Although there are many possible causes for this in the cartoon, here we represent three of them. Either they get a flat tire, they unexpectedly run out of gas, or there is some engine trouble that they have to deal with. For each of these scenarios there is a probability with which they happen. This probability we assume is 40% for a flat tire, 30% for being unexpectedly out of gas, and 60% for having an engine trouble. These are the probabilistic facts in our scenario, as each of these facts have a probability attached to them with which they occur. If any of these facts hold, it will lead to an adventure. This type of knowledge we can represent as a rule. Finally, we aim to query this scenario for the probability that an adventure will occur. </p>
In ProbLog the above scenario can be represented as follows:</p>
% Probabilistic facts:
</span>0</span>.</span>4</span>::flat_tire.
</span>0</span>.</span>3</span>::out_of_gas.
</span>0</span>.</span>6</span>::engine_trouble.
</span>
</span>% Rules:
</span>adventure_start </span>:- flat_tire.
</span>adventure_start </span>:- out_of_gas.
</span>adventure_start </span>:- engine_trouble.
</span>
</span>% Queries:.
</span>query</span>(adventure_start).
</span></code></pre>
As one can see, a ProbLog program is a combination of probabilistic facts, rules and queries (with comments in lines following %</code>). Probabilistic facts represent the facts of the domain with an attached probability between 0 and 1. Rules are deterministic rules (i.e. they have no probabilities attached) that show the system how new facts can be inferred from existing ones. Finally queries allow us to ask the program questions, such as the probabilities for a certain fact occuring. These program elements are similar to those employed in Prolog</a>, where a program consists of facts, rules and queries, with the main difference that there are probabilities attached to each fact.</p>
We can use infer new facts from these probabilistic facts using rules. For example, if we want to infer the probability of an adventure the query: query(adventure).</code> will return the probability 0.832</code>.</p>
Now we can make our Scooby Doo scenario a bit more complex. Suppose the group starts an adventure, after their van stopped working somehow, and they quickly realize that there is a mystery in the area. The location of this mystery is either an abandoned mansion, a local museum, an old theme park, or a nearby farm. We set the probabilities for each of these locations occurring at 0.3</code> for the abandoned mansion, 0.3</code> for a local museum, 0,2</code> for an old theme park and 0.2</code> for the nearby farm. We also assume that there is only one mystery location in each adventure. </p>
In order to express the requirements for the adventure locations succinctly, we make use of a feature called annotated disjunctions. This allows for a more readable way to state that only one of the stated choices holds true, with a given probability. Below is the ProbLog program extended to include this information.</p>
% Probabilistic facts:
</span>0</span>.</span>4</span>::flat_tire.
</span>0</span>.</span>3</span>::out_of_gas.
</span>0</span>.</span>6</span>::engine_trouble.
</span>
</span>0</span>.</span>3</span>::</span>monster_location</span>(abandoned_mansion); </span>0</span>.</span>3</span>::</span>monster_location</span>(local_museum); </span>0</span>.</span>2</span>::</span>monster_location</span>(old_theme_park); </span>0</span>.</span>2</span>::</span>monster_location</span>(nearby_farm).
</span>
</span>% Rules:
</span>adventure_start </span>:- flat_tire.
</span>adventure_start </span>:- out_of_gas.
</span>adventure_start </span>:- engine_trouble.
</span>
</span>adventure </span>:- </span>monster_location</span>(X), adventure_start.
</span>two_locations </span>:- </span>monster_location</span>(X), </span>monster_location</span>(Y), X \== Y.
</span>
</span>
</span>
</span>% Queries:.
</span>query</span>(two_locations).
</span></code></pre>
There are two other new concepts that we showcase here. One is using variables, notably the X</code> in monster_location(X)</code>, which helps to express that the values used for this variable all express the monster's location. The other is the use of restrictions in the use two_locations</code> to showcase that the probability for two monster locations occurring at once is 0. There are number of built-ins</a> that one can use for defining Problog models. In this case we use to define a rule to express that the two_locations</code> fact should be derived if there are two distinct monster locations. Given the example above, due to the use of an annotated disjunction for defining the monster location, the query query(two_locations).</code> will correctly probability of 0 for the chance of two monster locations at the same time.</p>
The final ingredient for a Scooby Doo story that we represent in this article is the monster. There are five types of monsters that can occur: a Mummy, a Zombie, a Ghost, a Swamp Monster and a Headless Horseman.  The chance at which these monsters occur is dependent on the current location. In the stories that we represent only 1 monster can occur in an adventure. </p>
Such cases can also be represented with annotated disjunctions, but they are now used as the head (which is the left hand side portion of the rule, with the :-</code> sign separating the two sides). This allows us to express the conditions, i.e. the monster locations, that is required for these facts. See our final example for the probabilities of monsters given the locations:</p>
% Probabilistic facts:
</span>0</span>.</span>4</span>::flat_tire.
</span>0</span>.</span>3</span>::out_of_gas.
</span>0</span>.</span>6</span>::engine_trouble.
</span>
</span>0</span>.</span>3</span>::</span>monster_location</span>(abandoned_mansion); </span>0</span>.</span>3</span>::</span>monster_location</span>(local_museum); </span>0</span>.</span>2</span>::</span>monster_location</span>(old_theme_park); </span>0</span>.</span>2</span>::</span>monster_location</span>(nearby_farm).
</span>
</span>
</span>% Rules:
</span>adventure_start </span>:- flat_tire.
</span>adventure_start </span>:- out_of_gas.
</span>adventure_start </span>:- engine_trouble.
</span>
</span>0</span>.</span>4</span>::</span>monster</span>(ghost); </span>0</span>.</span>4</span>::</span>monster</span>(vampire); </span>0</span>.</span>2</span>::</span>monster</span>(zombie) :- </span>monster_location</span>(abandoned_mansion).
</span>0</span>.</span>5</span>::</span>monster</span>(mummy); </span>0</span>.</span>2</span>::</span>monster</span>(headless_horseman); </span>0</span>.</span>3</span>::</span>monster</span>(ghost) :- </span>monster_location</span>(local_museum).
</span>0</span>.</span>5</span>::</span>monster</span>(zombie); </span>0</span>.</span>4</span>::</span>monster</span>(ghost); </span>0</span>.</span>1</span>::</span>monster</span>(mummy) :- </span>monster_location</span>(old_theme_park).
</span>0</span>.</span>4</span>::</span>monster</span>(ghost); </span>0</span>.</span>2</span>::</span>monster</span>(zombie); </span>0</span>.</span>2</span>::</span>monster</span>(headless_horseman); </span>0</span>.</span>2</span>::</span>monster</span>(vampire) :- </span>monster_location</span>(nearby_farm).
</span>
</span>adventure </span>:- </span>monster_location</span>(X), </span>monster</span>(Y), adventure_start.
</span>any_monster_location </span>:- </span>monster_location</span>(X).
</span>any_monster </span>:- </span>monster</span>(X).
</span>two_locations </span>:- </span>monster_location</span>(X), </span>monster_location</span>(Y), X \== Y.
</span>two_monsters </span>:- </span>monster</span>(X), </span>monster</span>(Y), X \== Y.
</span>vampire_after_flat_tire </span>:- </span>monster</span>(vampire), flat_tire.
</span>
</span>
</span>% Queries:.
</span>query</span>(adventure_start).
</span>query</span>(any_monster_location).
</span>query</span>(any_monster).
</span>query</span>(adventure).
</span>query</span>(two_locations).
</span>query</span>(two_monsters).
</span>query</span>(</span>monster</span>(ghost)).
</span>query</span>(vampire_after_flat_tire).
</span></code></pre>
In this example we also show a number of interesting facts that we might want to be able to query. The probability that an adventure starts is 0.832. Given that the probability for a monster location and a monster existing is 1 in both cases, and adventure requires only an "adventure start", a "monster" and its "location", the probability for an adventure happening is also 0.832. As mentioned before, due to the annotated disjunctions the probability of having two monsters or two locations is 0. We can also query for facts such as the probability of a monster occuring, which is inferred based on both the conditional probability given the monster location and the probability of the monster location itself. For example the probability of the monster being a ghost is 0.37. Finally we can calculate the probabilities for any particular scenario that we create, such as the probability of having the monster be a vampire after having a flat tire: 0.064. </p>
As one can see many spooky scenarios can be explored with ProbLog. Here we only taken a quick peek for representing a small portion of Scooby Doo stories, but these can also be applied to other domains, be it reasoning in the legal, financial, health and other fields. So do not be scared off and give it a try for any domain modelling you might encounter involving probabilities!</p>


Solving Financial Regulatory Compliance Using Software Contracts
2019-09-11T00:00:00+00:00
Abstract</h1>
"Semantic Web technologies have shown to have great potential in many different domains, to facilitate knowledge representation, exchange and reasoning, in a formal and yet both human and machine understandable way. In particular, within the health domain, they enable knowledge integration and understanding by explicitly defining and linking concepts and relationships using ontologies to information within clinical knowledge bases. This additional metadata also allows for automated decision support and semantic based analytics to be implemented, that facilitate improved healthcare at a lower cost. Unfortunately many existing datasets in healthcare environments are still stored in relational databases, as opposed to using semantic technologies. Due to this, the link with explicit metadata is often lacking or non-existent. Furthermore, both the databases and the clinical terminologies can be considerably large, making the mapping and subsequent uses of the information a difficult process. In a full fledged decision support system the level and accuracy of the mapping can greatly influence the effectiveness of any subsequent analysis and decision support tasks. This is especially true in clinical scenarios, where very large and complex sets of terms need to be mapped to relational databases. In this paper we aim to provide a general approach for interlinking relational data with clinical ontology based metadata that allows for a fine grade evaluation, with respect to the mapping's impact on analytics. We evaluate our approach by mapping information from clinical terminologies, such as SNOMED CT, to a large laboratory dataset contained in a relational database, with the goal of creating a full fledged, semantically enabled, analytics and decision support system."</p>
Authors</h1>
Newres Al Haider, Dilhan Thilakarathne, Joost Bosman</p>
Presented at</h1>
22ND INTERNATIONAL CONFERENCE ON APPLICATIONS OF DECLARATIVE PROGRAMMING AND KNOWLEDGE MANAGEMENT Declare 19
Link</a></p>
Related Project</h1>
Legal Banking Compliance</a></p>


Aesopica, Part 5: Blank Nodes
2019-06-08T00:00:00+00:00
This article is the fifth part of a series, examining the use of the Clojure language for representing Linked Data, using examples from Aesop's stories. The topic of this article is to explain the somewhat contentious subject of blank nodes. </p>
One of the strengths of RDF</a> as a graph representation format is the way resources are named. Through the use of Uniform Resource Identifiers (URIs)</a> every element of the graph can be uniquely identified. With such generic and powerful facilities for naming it can easily represent information in any domain. For example on schema.org</a> elements have been defined for the notion of a medical condition (http://schema.org/MedicalCondition)</a>, employee (http://schema.org/employee)</a> and bank account (http://schema.org/BankAccount)</a>, just to name a few. In particular the namespacing part of the URI, e.g.: http://schema.org/ for http://schema.org/employee , helps to ensure that concepts can be uniquely named, even in scenarios with multiple definitions of the same concept. </p>
Blank nodes go against this notion of making everything explicitly named. In fact an alternative name for a blank node is "an anonymous resource"". Instead of giving a resource an explicit name with a URI a placeholder is used. This indicates the existence of the resource, but does not tie it together with a namespaced name. In the Turtle syntax for RDF we can use a label prefixed by _:</code> to indicate a blank node. For example, the following RDF graph states that the fox and the stork both give out an invitation.</p>
@base </span><http://www.newresalhaider.com/ontologies/aesop/foxstork/></span> .
</span>
</span><#fox> <#gives-invitation> </span>_:</span>invitation1.
</span><#stork> <#gives-invitation> </span>_:</span>invitation2.
</span></code></pre>
Due to the invitations having different names we can expect them to be different resources. However the exact names of these resources do not matter. For example the RDF triples below have arguably the same meaning: </p>
@base </span><http://www.newresalhaider.com/ontologies/aesop/foxstork/></span> .
</span>
</span><#fox> <#gives-invitation> </span>_:</span>abc.
</span><#stork> <#gives-invitation> </span>_:</span>xyz.
</span></code></pre>
It is important to reiterate that the blank nodes are not resource identifiers such as URIs. They are also only local in scope: an _:invitation1</code> in one graph and a _:invitation1</code> in another are not referring to the same thing. Even resources used in the similar places in different graphs are not the same. For example the _:abc</code> and the  _:invitation1</code> used in the above graphs, while expressing the same meaning, are not the same resource.</p>
The above features are both the strength and the weakness of using anonymous resources. We are not required to use a specific named identifier, but this makes referring to resources and comparing them more difficult. </p>
There are number of scenarios where such anonymous resources can be useful. For example, when representing complex structures not easily expressed in triples where we would need "placeholder nodes" in the graph but do not particularly care about its naming. In other cases we want to hide some information, and blank nodes could be used as placeholders for named resources. A good overview of the various uses of Blank Nodes can be found in the paper: On Blank Nodes</a>.</p>
With the above description of blank nodes, we also want to provide something similar in our Clojure based RDF representation as well. As we are using keywords to represent URIs in our Clojure representation (i.e. :fox and :rdf/type), a natural element to differentiate them is to use symbols for blank nodes. Symbols are created in Clojure by prefixing it with '</code>. For example in the Clojure based RDF representation we use blank nodes for representing invitations:</p>
(</span>def </span>fox-and-stork-blank-node-edn
</span>  {</span>::aes/context
</span>   {</span>nil </span>"</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/</span>"
</span>    </span>:rdf </span>"</span>http://www.w3.org/1999/02/22-rdf-syntax-ns#</span>"}
</span>   </span>::aes/facts
</span>   #{[</span>:fox :rdf/type :animal</span>]
</span>     [</span>:stork :rdf/type :animal</span>]
</span>     [</span>:fox :gives-invitation </span>'invitation1]
</span>     ['invitation1 </span>:has-invited :stork</span>]
</span>     ['invitation1 </span>:has-food :soup</span>]
</span>     ['invitation1 </span>:serves-using :shallow-plate</span>]
</span>     [</span>:stork :gives-invitation </span>'invitation2]
</span>     ['invitation2 </span>:has-invited :fox</span>]
</span>     ['invitation2 </span>:has-food :crumbled-food</span>]
</span>     ['invitation2 </span>:serves-using :narrow-mouthed-jug</span>]
</span>     [</span>:fox :can-eat-food-served-using :shallow-plate</span>]
</span>     [</span>:fox :can-not-eat-food-served-using :narrow-mouthed-jug</span>]
</span>     [</span>:stork :can-eat-food-served-using :narrow-mouthed-jug</span>]
</span>     [</span>:stork :can-not-eat-food-served-using :shallow-plate</span>]}})
</span></code></pre>
The above is our version of "The Fox and the Stork" story that we previously explored in this series of articles The main difference is that two blank nodes are used:  'invitation1</code> and 'invitation2</code>, instead of named resources. Perhaps the story teller might want to hide some details of their invitations.</p>
As with all the previous features in this series of articles, blank nodes been implemented in our implementation of this syntax in the Aesopica</a> library for using Clojure to write Linked Data. Hiding details with blank nodes, even in Clojure, is now just one library away.</p>


Exploring the Deep, Part 1: Introduction
2019-02-04T00:00:00+00:00
Deep Learning is a field within Artificial Intelligence (AI) that has got quite a lot of attention lately, due to some truly impressive results in recent years. From recognizing objects in images with accuracy that rivals humans, to generating realistic looking texts, to even beating professional players in Starcraft 2</a>, some truly groundbreaking applications are done with Deep Learning techniques.</p>
As someone whose AI background is more Symbolic Artifical Intelligence</a>, but is anxious to learn, it is a good time as any to explore this field. Thankfully there are some excellent books and tutorials out there to get a nice start. </p>
One book that has caught my attention is Dive into Deep Learning</a>, which seems to have a nice overview of the field, while being very practical at the same time. I aim to work through the book and share my notes, summaries and experiences on this journey in a series of articles. My goal is to write these articles for a bit more broader audience than the book itself. I hope that even those with less exposure to the field than me could follow along, and if a topic piques their interest, they can use the book for a more detailed reference point. </p>
The first chapter</a> that I aim to go over is the introduction to the book as well as Machine Learning in general. As Deep Learning (DL), is a form of Machine Learning (ML), an intro/refresher on the topic of Machine Learning is a natural start.</p>
What is Machine Learning?</h2>
In most scenarios, programs are written in way where the programmer gives precise information about the problem and/or how to solve it. For example in imperative programming</a> the program is essentially a list of statements that command the computer what to do. Even in the many forms of declarative programming</a>, such as functional-</a> or logic programming</a>, providing the solution is done by explicit descriptions by the coder. The coder would describe the entirety of the problem with mathematical functions or with logical axioms respectively.</p>
However there are scenarios where such paradigms are not adequate. Take for example a case where we where have set of photos of cats and dogs. Suppose we want need to write a program that, given a photo, can recognize the species and breed of the animal depicted. Generally it is difficult to even begin formulating such a problem in terms of the usual programming paradigms. Usually any handwritten solution we can come up with will be very brittle: it will often fail to recognize the right kind of pet from the photo, or it would only be correct for a very limited set of photos. </p>
An alternative way to solve this problem is by letting the computer learn what kind of pet is in the photo. The computer can use a set of labelled examples where the kind of animal and breed is already known and given alongside the photo. In such cases we start with an initial, perhaps even random, model that can make this prediction. Such a model will likely be pretty bad at the start, otherwise the problem would already be solved. Then we use the labelled data to update this model. The hope is that the new model will be better at classifying animals. The intention is to keep adding data and improving the model until we decide that our model is good enough to solve our pet classification problem.</p>
This learning from previous experiences by the machine to solve a problem is denoted as machine learning</em>. </p>

  
  
      The recognition of dogs and cats, along with their breeds, is actually an interesting research problem in machine learning. This is due to the subtle differences between breeds making the problem difficult for computers. These images are from the  The Oxford-IIIT Pet Dataset</a> that can be used to help evaluate techniques aiming to tackle this issue. The images of this dataset are licensed under  Creative Commons Attribution-ShareAlike 4.0 International License.</a> Copyright of the images is with their original owner.</h4>
  </figcaption>
</figure>
There are a large number of forms of machine learning, of which deep learning is only a particular branch, but there are common elements among the various techniques.</p>
Elements of Machine Learning</h2>
Data</h3>
Data is the example information from which the machine learning aims to learn. In the previous scenario, the data is the labelled photos of pets.</p>
Model</h3>
The model designates the whole process of using the data and transforming it into the goal of machine learning. In our example the model would be the full process that can take a given photo and transform it into a prediction of the pet depicted.</p>
Objective Function</h3>
The objective function denotes how good, or bad, our model currently is. As mentioned before, the reason we perform machine learning is that we aim to let the computer learn how to solve a problem. The objective function allow us to measure the quality of our current solution, so it can be assessed whether the machine learning is progressing towards an acceptable solution.</p>
Optimization Algorithm</h3>
The optimization algorithm that describes the process of moving from one model to a better one. This allows for the learning to happen by moving from one model to a one that better solves the problem.</p>
Kinds of Machine Learning</h2>
Machine learning is a large field with a wide variety of techniques and potential problems to solve. Generally techniques are differentiated by the data they input and output, as well as the specific type of problem they target.</p>
Supervised Learning</h3>
Supervised learning aims to predict targets given some input data. A good example of this is our previous scenario where we were trying to predict what type of pet is in a photo. If we have a number of photos for which we know what kind of animal is depicted in them (i.e. labeled data), we could perform such supervised learning.</p>
Various kinds of supervised learning can be further subdivided based on what kind of target we aim to predict. Some common types of supervised learning are:</p>
Classification</h4>
The above example is actually called a classification</em> scenario. In general, classification problems are those where given an example, we aim to find to what particular class that example belongs to. While in the example we described the classes as the species and the breeds of pets, many other types of classes can be used depending on the problem we aim to solve.</p>
Tagging</h4>
Consider a scenario where there could be more than one pet in the photo and we want to recognize each of them. Predicting in such cases is called tagging</em>. The main difference between classification and tagging, is that with tagging multiple classes need to be recognized at once (e.g.: the photo can have both a dog and a cat ). This in contrast with classification, where the classes are exclusive (e.g.: dog or a cat).</p>
Regression</h4>
The case where we want to predict a (real valued) number, for example the weight of the pet depicted in the photo, is called a regression</em>.</p>
Search and Ranking</h4>
Search and ranking is a scenario where we want to figure out an order between various items. For example suppose that we want to figure out popularity rankings between various breeds of dogs and cats.</p>
Unsupervised Learning</h3>
In cases with supervised learning, a set of examples for which the result is already known must be available. Often large quantities of such information is required to get accurate results. However such information can be scarce, or simply unavailable. </p>
Even in such cases there is a wealth of information that can still be learned from the data through machine learning. Some example techniques include:</p>
Clustering</h4>
With the absence of clearly labelled categories given to us, we can still want to group the data into categories. This process is called clustering</em>. For example suppose we have a set of photos depicting all sort of pets, without any labels. Here we could still aim to group similar pets into clusters.</p>
Data Generation/Synthesis</h4>
Given a set of data, we can also aim to synthesize data that is similar to the given data. For example if we have a large set of dog photos we can also aim to generate a photo of a, non-existent, dog.</p>
Reinforcement Learning</h3>
The final group of machine learning techniques that we go over in this article are those where the learning process is interacting with the environment. In reinforcement learning the main goal is to figure out what kind of action needs to be taken in a particular situation. As an example, take a scenario where you want to teach a robot dog to play football. </p>
There process of reinforcement learning is as follows. The agent interacts with the environment through actions and observes the environment reacting to those actions. Based on the environment the agent may get rewards, either positive or negative. The rewards, together with the observations, guide the selection of subsequent actions. There is a cycle of actions, leading to observations and potentially rewards, and then again to actions. This cycle can then be repeated possibly indefinitely or until some goal is reached. </p>
To make this description a bit more concrete, lets take our example scenario of letting a robot dog learn football. Here the agent is the decision making process of the robot dog. The environment itself is a playing field with other robots and a ball. The observations are the sensor data of this environment, such as the vision of the robot. Rewards could be given for taking control of the ball, making a successful pass and of course scoring. Negative rewards can be given for the robot missing a pass, walking off field and other undesirable actions.</p>

  
  
      Teaching four legged robots, such as the  AIBO robot dogs</a>, to play football is one of the leagues that is part of the RoboCup</a> competition aimed at promoting robotics and AI research. This photo depicts the  team in a four-legged league game from RoboCup 2006 in Breman, Germany. Public Domain Photo by Brad Hall</a></h4>
  </figcaption>
</figure>
Reinforcement learning as a whole can be pretty complex with lots of variables based on can be known about the environment, state or actions. Subcategories of reinforcement are generally based on the given restrictions with these variables.</p>
Towards Deep Learning</h2>
The above is a general introduction to machine learning, so the question arises how deep learning fits into the picture. Machine learning as a field has quite a bit of history with many of its techniques being laid out in the previous century. A number of such techniques, such as Neural Networks</a> initially showed great promise but research and applications on them were languishing due to the lack data and large amount of processing power to enable such techniques. In recent years this has changed, alongside with some important advances, that led to deep learning techniques being one of the most effective approaches for various machine learning applications. </p>
In addition the availability of frameworks for deep learning, such as MXNet</a>, which we will use in our learning process, means that the field is very inviting for new users and applications. </p>
Conclusion</h2>
This was a quick rundown of the first chapter. The book actually goes into much more detail, with some previews of techniques and extensive references, for the interested reader. If you have any remarks or suggestions for this series of articles please let me know! I am quite anxious for the next part when I hope to plunge deep into some coding with MXNet</a>. </p>


Prolog and the Price of Peace
2019-02-01T00:00:00+00:00
The use of logic is a common element in Science Fiction. In the Star Trek universe Vulcans</a> are a species that is famously known for aiming to live by logic and reason. Because of this they are often considered masters of these subjects. Yet there was a case in an episode of the Star Trek show Deep Space 9</a>, where Quark</a>, a Ferengi</a>, was able to convince Sakonna, a Vulcan, of the error of her logic and reasoning. </p>
For this Quark made use of Ferengi philosophy, namely the Third Rule of Acquisition. The Rules of Acquisition are a series of proverbs and guidelines that govern Ferengi society, and notably their business dealings that take a prominent place in their lives. The Third Rule of Acquisition states that "Never spend more for an acquisition than you have to." By applying this rule to the situation in the episode Quark was able make it clear to Sakonna that the best time broker a peace agreement would be "right now", as the price of peace is at an all time low. </p>

  
  
      Quark explaining the Third Rule of Acquisition to Sakonna Copyright CBS Corporation.</h4>
  </figcaption>
</figure>

Much like with Ferengi society, various rules and regulations play a prominent role in our lives and dealings with each other. So much so, that navigating the various rules to their logical conclusion, whether in the realm of law, finance and other domains, is often a difficult process. Thankfully there are tools and techniques to help us. The programming language Prolog, in particular, can be a very helpful in dealing with various rules and logical problems. This article aims to provide a brief introduction to this language using by using it to show how Quark's reasoning can be implemented within a computer program.</p>
Prolog is a logic programming language, originally created in the 1970s, but with many modern implementations such as SWI-Prolog</a>. With logic programming programs are written and solved using some variation of a formal logic. Using such logic, information about the problem and its domain is first declared. Then the user can pose queries about the problem domain, which the programming language aims to answer through reasoning with the available information. This style of programming contrasts with the more common, imperative, paradigm. Instead of telling the computer how to solve the problem, we can declare information about the problem and let the computer, through the use of logic, solve it for us.</p>
In order to show how Prolog works, we aim to use it to represent the same problem and reasoning that Quark used to convince Sakonna. To do this first lets examine the situation depicted the episode a bit more closely. In the episode the Maquis are a group that are at odds Cardassian colonists both living the Demilitarized Zone. Tensions were escalating as the Cardassian colonists were recently supplied in secret with weapons by the Cardassian Union. As a result Sakonna, along with other members of the Maquis, aimed at acquiring more weapons of their own. Even after it was found that the Cardassian Union was behind weapon supplies, Sakonna still aimed at gathering more, as a way to ensure peace. Quark pointed out the flaws in the logic of her argument, given the Third Rule of Acquisition: "Never spend more for an acquisition than you have to." If the goal of Sakonna is truly to acquire peace, than it is the perfect time to be negotiating with the Cardassian colonists. With the flow of the smuggled in weapons having stopped, and both sides already having weapons, neither the Maquis nor the Cardassian colonists have any advantage. By aiming to acquire weapons still, they would only escalate the conflict, and making peace more costly in the long run.</p>
To express this scenario with Prolog, we are going to introduce some elements of this language first (for a bit more through introduction see the excellent Learn Prolog Now!</a> available online). There are three basic elements to a Prolog program: facts, rules and queries. </p>
Facts are elements that have been stated to hold. For example the following statement:</p>
advantage</span>(cardassians).
</span></code></pre>
can be used to denote that the advantage is held by the Cardassians colonists.</p>
Rules are a way for Prolog to infer new information from the knowledge that already exists. For example if we aim to state that "if the Cardassian colonists have the advantage the price of peace is high" in Prolog we would have the rule: </p>
priceOfPeace</span>(high) :- </span>advantage</span>(cardassians).
</span></code></pre>
There are two parts to writing such rules in Prolog: the body and the head. The body of the rule is in this case advantage(cardassians)</code> while the head is priceOfPeace(high)</code>. In Prolog if the body of the rule holds true, than it can be concluded that the head is true as well. This exactly matches to what we aim to express: if the Prolog program knows that the Cardassian colonists hold the advantage, than it can conclude that the price of peace is high. </p>
Facts and rules together form the knowledge base that can describe a domain. In this case the domain is the situation between the Cardassian colonists and the Maquis. The final piece, queries, allows us to examine this knowledge base and ask questions on what Prolog can infer from this knowledge. For the above example we could query whether the price of peace is indeed high, which should follow directly from the semantics of the rules and facts that we have described.</p>
Given that we load in the knowledge base consisting of the above-mentioned fact and rule, the query: </p>
?-  </span>priceOfPeace</span>(high).
</span></code></pre>
that asks whether the price of peace is high, will return true</code> indicating that this is indeed the case. Instead if we ask whether the price of peace is low, using the query:</p>
?-  </span>priceOfPeace</span>(low).
</span></code></pre>
the result will be false</code> indicating this is not the case.</p>
We can also ask more open ended questions using variables. Variables are written by starting with a capitalized letter (or an underscore), such as X</code>. These variables could be used to write queries with unknowns. The query: </p>
?-  </span>priceOfPeace</span>(X).
</span></code></pre>
will give us the result: </p>
X = high
</span></code></pre>
Which again is what would directly follow from the facts and the rules. What Prolog does with variables is to try to "unify" them with known values that match (or other variables). In this case from the knowledge base we have given it, it unifies it with the value of high</code>. </p>
From the basic elements of facts, rules and queries Prolog is able to represent and answer problems in many domains. This is also shows off the feature of a logical programming language, such as Prolog, that make it different compared to many other programming languages. Instead of telling the program what to do, the program becomes a description of the domain, over which queries are answered to solve the overall goal. </p>
Beyond the above basics, Prolog has many features to support the description of more complicated domains. Here we only list a few that helps us to convey our translation of Quark's reasoning into Prolog. </p>
One such feature is the use variables within the rules themselves. The rule </p>
acquirePeace</span>(X) :- </span>priceOfPeace</span>(X).
</span></code></pre>
indicates that the price of acquiring peace depends on the price of peace.</p>
The rule </p>
priceOfPeace</span>(low)  :- </span>advantage</span>(cardassians), </span>advantage</span>(maquis).
</span></code></pre>
shows off the logical operator and</strong>, meaning that the price of peace is low if both the Cardassians and the Maquis have the advantage. </p>
The rule</p>
priceOfPeace</span>(low)  :- \+ </span>advantage</span>(cardassians), \+ </span>advantage</span>(maquis).
</span></code></pre>
shows off the logical not</strong>, denoted by the '+' symbol, which states that the price of peace is also low if neither the Cardassians nor the Maquis have the advantage.</p>
Finally, much like in other languages, comments for the code can also be written. This is done by prefacing a line with "%%".</p>
Now if we put every element together to use Prolog to describe Quarks scenario, a possible knowledge base is as follows:</p>
%% Rules
</span>acquirePeace</span>(X)  :- </span>priceOfPeace</span>(X).
</span>
</span>priceOfPeace</span>(low)  :- \+ </span>advantage</span>(cardassians), \+ </span>advantage</span>(maquis).
</span>priceOfPeace</span>(low)  :- </span>advantage</span>(cardassians), </span>advantage</span>(maquis).
</span>priceOfPeace</span>(high) :- </span>advantage</span>(cardassians).
</span>priceOfPeace</span>(low) :- </span>advantage</span>(maquis).
</span>
</span>
</span>advantage</span>(cardassians)  :- </span>weapons</span>(cardassians), \+ </span>weapons</span>(maquis).
</span>advantage</span>(maquis)  :- \+ </span>weapons</span>(cardassians), </span>weapons</span>(maquis).
</span>
</span>%% Facts
</span>weapons</span>(cardassians).
</span>weapons</span>(maquis).
</span></code></pre>
In the overall knowledge base, the logic of Quark's reasoning is broken down into a number of rules and a pair of facts. </p>
The rules state that advantage of either side, the Cardassians settlers or the Maquis, is decided by one side having weapons while the other side has not. The price of peace is in turn dependent on who has the advantage. In this formalisation the price of peace is only high if Cardassians have the advantage. Finally the price of acquiring peace is dependent solely on the price of peace.</p>
The facts for this scenario is that both the Cardassian settlers and the Maquis have weapons. </p>
With this knowledge base describing Quark's reasoning process the query to see for what price we can aquire peace:</p>
?-  </span>acquirePeace</span>(X).
</span></code></pre>
will indeed return</p>
X = low
</span></code></pre>
indicating that acquiring peace can be done at a low price. </p>
Going a bit beyond this query, if we would further want to test the logic of Quark's scenario, we can also inquire about other information.</p>
For example if we would ask for the price of peace:</p>
?-  </span>priceOfPeace</span>(X).
</span></code></pre>
we would also unsurprisingly get </p>
X = low
</span></code></pre>
as with the rules defined in our knowledge base these values are intertwined. </p>
We can also ask for who has the advantage:</p>
?-  </span>advantage</span>(X).
</span></code></pre>
which will return: </p>
false</span>.
</span></code></pre>
which is completely in line with the scenario that noone has the advantage. </p>
We can also ask who has weapons, the query of:</p>
?-  </span>weapons</span>(X).
</span></code></pre>
gives us the answer</p>
X = cardassians
</span></code></pre>
... but wait, this can not be right! Did we not state in the knowledge base that both the Cardassian colonists and the Maquis have weapons? Prolog can actually return mutliple answers, if they exist. In SWI-Prolog</a> this can be done by pressing semicolon (;</code>) after an answer, in which case another answers is returned, if it exists or simply false otherwise. By pressing ;</code> once after the first answer we will see a total output such as:</p>
X = cardassians </span>;
</span>X = maquis.
</span></code></pre>
Hopefully Sakonna would also be quite convinced with the logic of Prolog, but just to be absolutely sure, lets examine some scenario's where the facts of the scenario are changed. In such cases the rules should still be applied in ways that we would expect.</p>
Suppose that our only fact now is that only the cardassians colonists have weapons, while rules remain the same. Giving the query for knowing the price of acquiring peace:</p>
?-  </span>acquirePeace</span>(X).
</span></code></pre>
will now return:</p>
X = high 
</span></code></pre>
Hopefully, much like Quark's application of the Third Rule of Acquisition, this intro would have been enough to convince Sakonna of the logic of purchasing peace at the lowest possible price. Similarly, I hope that as a reader, this article gave some insight to the usefulness of Prolog in such scenarios. Next time, you are dealing with a problem domain, that would easily match to a similar scenario of rules, facts and queries, give Prolog a try. It is, after all, a very logical choice.</p>


Aesopica, Part 4: Basic Conversion to Other Formats
2018-12-18T00:00:00+00:00
This article is the fourth part of a series, examining the use of the Clojure language for representing Linked Data, with examples from Aesop's stories. The topic of this article is to describe how to do some basic conversions from our Clojure representation of Linked Data, to some of the other formats, such as Turtle</a>, NQUADS</a> or JSON-LD</a>. </p>
In previous articles of this series, we created a Clojure based syntax for defining Linked Data. In order to make this syntax a viable member of the Linked Data ecosystem, it is important to provide conversion functionality to other Linked Data formats. This allows for the user of the associated Aesopica</a> library, to create and use Linked Data in a Clojure based environment, and convert it, when needed, to other formats. In order to implement this functionality we made use of Clojure's Java interop and the Apache Jena</a>, and made it available in the Aesopica</a> library.</p>
To start off we begin with the basic example of "The Fox and The Stork" story that we introduced initially</a>. The Clojure representation of this is as follows:</p>
(</span>def </span>fox-and-stork-edn
</span>  {</span>::aes/context
</span>   {</span>nil </span>"</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/</span>"
</span>    </span>:rdf </span>"</span>http://www.w3.org/1999/02/22-rdf-syntax-ns#</span>"}
</span>   </span>::aes/facts
</span>   #{[</span>:fox :rdf/type :animal</span>]
</span>     [</span>:stork :rdf/type :animal</span>]
</span>     [</span>:fox :gives-invitation :invitation1</span>]
</span>     [</span>:invitation1 :has-invited :stork</span>]
</span>     [</span>:invitation1 :has-food :soup</span>]
</span>     [</span>:invitation1 :serves-using :shallow-plate</span>]
</span>     [</span>:stork :gives-invitation :invitation2</span>]
</span>     [</span>:invitation2 :has-invited :fox</span>]
</span>     [</span>:invitation2 :has-food :crumbled-food</span>]
</span>     [</span>:invitation2 :serves-using :narrow-mouthed-jug</span>]
</span>     [</span>:fox :can-eat-food-served-using :shallow-plate</span>]
</span>     [</span>:fox :can-not-eat-food-served-using :narrow-mouthed-jug</span>]
</span>     [</span>:stork :can-eat-food-served-using :narrow-mouthed-jug</span>]
</span>     [</span>:stork :can-not-eat-food-served-using :shallow-plate</span>]}})
</span></code></pre>
The full details of this representation are explained in our previous work. Here we only briefly summarize its main elements. The above mentioned example defines a knowledge base containing a set of facts that describe the "The Fox and The Stork" story. Each fact is a triple of a subject, predicate and object. These elements are all represented by an URI</a>, but for human readability and use as well as ease of use in Clojure, they are identified by (namespaced) keywords. The context map, containing the keywords for the namespaces used, allows us to transform the namespaced keywords into full URIs when required. Of course there are more elements possible in Linked Data/RDF and in this representation as well, such as literal values</a>, but this summary should suffice for this article.</p>
Now in order to explain how this conversion is done, assuming no familiarity with Clojure or a similar language, two new concepts are required. </p>
First, it is important to note that in the above example we bind the "The Fox and The Stork", Linked Data representation to the fox-and-stork-edn</code> variable. Although this is not a necessity for creating the knowledge base, it allows us to reuse this definition from inside the code and in this article as well, without explicitly writing out the full representation each time. </p>
The second concept that we make use is how Clojure functions are called to be executed. In Clojure invoking a function has the general form of (function-name param1 param2 ...)</code>. For example lets assume that the function for translation from our Clojure representation to Turtle is represented by conv/convert-to-turtle</code>. Here conv</code> is a shorthand for Aesopica's conversion namespace. Given this, the call to translate to a Turtle string representation of the Linked Data can be invoked by:</p>
(</span>conv/convert-to-turtle</span> fox-and-stork-edn)
</span></code></pre>
The resulting string representation shows the same Linked Data knowledge base in Turtle syntax:</p>
"</span>@base <http://www</span>.</span>newresalhaider</span>.</span>com/ontologies/aesop/foxstork/> </span>.
</span>@prefix rdf: <http://www</span>.</span>w3</span>.</span>org/1999/02/22-rdf-syntax-ns#> </span>.
</span>
</span><fox> rdf:type <animal></span>.
</span><stork> rdf:type <animal></span>.
</span><fox> <gives-invitation> <invitation1></span>.
</span><invitation1> <has-invited> <stork></span>.
</span><invitation1> <has-food> <soup></span>.
</span><invitation1> <serves-using> <shallow-plate></span>.
</span><stork> <gives-invitation> <invitation2></span>.
</span><invitation2> <has-invited> <fox></span>.
</span><invitation2> <has-food> <crumbled-food></span>.
</span><invitation2> <serves-using> <narrow-mouthed-jug></span>.
</span><fox> <can-eat-food-served-using> <shallow-plate></span>.
</span><fox> <can-not-eat-food-served-using> <narrow-mouthed-jug></span>.
</span><stork> <can-eat-food-served-using> <narrow-mouthed-jug></span>.
</span><stork> <can-not-eat-food-served-using> <shallow-plate></span>.</span>"
</span></code></pre>
This representation has a similar form to our Clojure based notation. A set of facts is represented and prefixes and/or a base prefix is used, to enable easy reading and writing of the triples.</p>
Now let us look at some other formats and conversions. </p>
TriG</a> is an extension of the Turtle format for enabling "named graphs". Now this is a topic of a previous article</a> but here it is suffice to say that by associating a set of facts with a specific graph, we enable the easy adding of metadata to these facts. To show this conversion, we use an example that uses this notion:</p>
(</span>def </span>fox-and-stork-named-graph-edn
</span>  {</span>::aes/context
</span>   {</span>nil </span>"</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/</span>"
</span>    </span>:rdf </span>"</span>http://www.w3.org/1999/02/22-rdf-syntax-ns#</span>"
</span>    </span>:time </span>"</span>http://www.w3.org/2006/time#</span>"}
</span>   </span>::aes/facts
</span>   #{[</span>:fox :rdf/type :animal</span>]
</span>     [</span>:stork :rdf/type :animal</span>]
</span>     [</span>:fox :gives-invitation :invitation1 :dinner1</span>]
</span>     [</span>:invitation1 :has-invited :stork :dinner1</span>]
</span>     [</span>:invitation1 :has-food :soup :dinner1</span>]
</span>     [</span>:invitation1 :serves-using :shallow-plate :dinner1</span>]
</span>     [</span>:stork :gives-invitation :invitation2 :dinner2</span>]
</span>     [</span>:invitation2 :has-invited :fox :dinner2</span>]
</span>     [</span>:invitation2 :has-food :crumbled-food :dinner2</span>]
</span>     [</span>:invitation2 :serves-using :narrow-mouthed-jug :dinner2</span>]
</span>     [</span>:invitation1 :serves-using :narrow-mouthed-jug :dinner2</span>]
</span>     [</span>:dinner1 :time/before :dinner2</span>]}})
</span></code></pre>
As one can see, here we simply extend our triple based representation of facts to include either triples or quads. In a quad the last element is the graph name identifier of the graph the fact is a member of.</p>
Translating this representation to TriG can be done by:</p>
(</span>conv/convert-to-trig</span> fox-and-stork-named-graph-edn)
</span></code></pre>
Which results in the following string representation that is TriG formatted: </p>
{ </span><http://www.newresalhaider.com/ontologies/aesop/foxstork/fox>
</span>        </span><http://www.w3.org/1999/02/22-rdf-syntax-ns#type>  <http://www.newresalhaider.com/ontologies/aesop/foxstork/animal></span> .
</span><http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner1>
</span>        </span><http://www.w3.org/2006/time#before>  <http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner2></span> .
</span><http://www.newresalhaider.com/ontologies/aesop/foxstork/stork>
</span>        </span><http://www.w3.org/1999/02/22-rdf-syntax-ns#type>  <http://www.newresalhaider.com/ontologies/aesop/foxstork/animal></span> .
</span>}
</span>
</span><http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner2></span> {
</span>    </span><http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation2>
</span>            </span><http://www.newresalhaider.com/ontologies/aesop/foxstork/serves-using>  <http://www.newresalhaider.com/ontologies/aesop/foxstork/narrow-mouthed-jug></span> ;
</span>            </span><http://www.newresalhaider.com/ontologies/aesop/foxstork/has-food>  <http://www.newresalhaider.com/ontologies/aesop/foxstork/crumbled-food></span> ;
</span>            </span><http://www.newresalhaider.com/ontologies/aesop/foxstork/has-invited>  <http://www.newresalhaider.com/ontologies/aesop/foxstork/fox></span> .
</span>    </span><http://www.newresalhaider.com/ontologies/aesop/foxstork/stork>
</span>            </span><http://www.newresalhaider.com/ontologies/aesop/foxstork/gives-invitation>  <http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation2></span> .
</span>    </span><http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation1>
</span>            </span><http://www.newresalhaider.com/ontologies/aesop/foxstork/serves-using>  <http://www.newresalhaider.com/ontologies/aesop/foxstork/narrow-mouthed-jug></span> .
</span>}
</span>
</span><http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner1></span> {
</span>    </span><http://www.newresalhaider.com/ontologies/aesop/foxstork/fox>
</span>            </span><http://www.newresalhaider.com/ontologies/aesop/foxstork/gives-invitation>  <http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation1></span> .
</span>    </span><http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation1>
</span>            </span><http://www.newresalhaider.com/ontologies/aesop/foxstork/has-food>  <http://www.newresalhaider.com/ontologies/aesop/foxstork/soup></span> ;
</span>            </span><http://www.newresalhaider.com/ontologies/aesop/foxstork/serves-using>  <http://www.newresalhaider.com/ontologies/aesop/foxstork/shallow-plate></span> ;
</span>            </span><http://www.newresalhaider.com/ontologies/aesop/foxstork/has-invited>  <http://www.newresalhaider.com/ontologies/aesop/foxstork/stork></span> .
</span>}
</span>
</span></code></pre>
There are a number of differences in representation from the above TriG output to he Clojure representation, but also from the previous Turtle output. Probably one of the most apparent is that in this output no prefixes are used: URIs are all written out fully. Both Turtle and TriG are flexible in whether they abbreviate URIs with prefixes or not. This is completely left up to the author, on in this case the specific way the conversion has been implemented. Another difference is how graphs are identified. Instead of using a quad like formatting for denoting the graph to which each fact belongs to they are grouped together. For example in the form of: <http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner2> { ... }</code>, all the facts inside the curly braces belong to the <http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner2></code> graph. Finally a somewhat similar construction is used to abbreviate a group of triples that all use the same object. Instead of writing each fact out fully, "predicate-lists" are used to match a single subject with a series subject and object pairs. This is quite a nice feature, and something similar is definitely on the list of future improvements to the Clojure notation, although care must be taken that such shorthands can make the definition a bit more complex. </p>
Speaking of complexity, an interesting format created with the purpose of being very simple is NQUADS</a>. This is a straightfoward, line based syntax where each fact is represented by a single line. It is actually an extension of the N-Triples</a> format, with support added for handling named graphs. The conversion of our named graph example using the function invocation:</p>
(</span>conv/convert-to-nquads</span> fox-and-stork-named-graph-edn)
</span></code></pre>
would give us the following example:</p>
<http://www.newresalhaider.com/ontologies/aesop/foxstork/fox> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://www.newresalhaider.com/ontologies/aesop/foxstork/animal></span> .
</span><http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner1> <http://www.w3.org/2006/time#before> <http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner2></span> .
</span><http://www.newresalhaider.com/ontologies/aesop/foxstork/stork> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://www.newresalhaider.com/ontologies/aesop/foxstork/animal></span> .
</span><http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation2> <http://www.newresalhaider.com/ontologies/aesop/foxstork/serves-using> <http://www.newresalhaider.com/ontologies/aesop/foxstork/narrow-mouthed-jug> <http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner2></span> .
</span><http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation2> <http://www.newresalhaider.com/ontologies/aesop/foxstork/has-food> <http://www.newresalhaider.com/ontologies/aesop/foxstork/crumbled-food> <http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner2></span> .
</span><http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation2> <http://www.newresalhaider.com/ontologies/aesop/foxstork/has-invited> <http://www.newresalhaider.com/ontologies/aesop/foxstork/fox> <http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner2></span> .
</span><http://www.newresalhaider.com/ontologies/aesop/foxstork/stork> <http://www.newresalhaider.com/ontologies/aesop/foxstork/gives-invitation> <http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation2> <http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner2></span> .
</span><http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation1> <http://www.newresalhaider.com/ontologies/aesop/foxstork/serves-using> <http://www.newresalhaider.com/ontologies/aesop/foxstork/narrow-mouthed-jug> <http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner2></span> .
</span><http://www.newresalhaider.com/ontologies/aesop/foxstork/fox> <http://www.newresalhaider.com/ontologies/aesop/foxstork/gives-invitation> <http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation1> <http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner1></span> .
</span><http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation1> <http://www.newresalhaider.com/ontologies/aesop/foxstork/has-food> <http://www.newresalhaider.com/ontologies/aesop/foxstork/soup> <http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner1></span> .
</span><http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation1> <http://www.newresalhaider.com/ontologies/aesop/foxstork/serves-using> <http://www.newresalhaider.com/ontologies/aesop/foxstork/shallow-plate> <http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner1></span> .
</span><http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation1> <http://www.newresalhaider.com/ontologies/aesop/foxstork/has-invited> <http://www.newresalhaider.com/ontologies/aesop/foxstork/stork> <http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner1></span> .
</span></code></pre>
As one can see this format does not use prefixes: each fact is a triple or a quad on a single line ending with a dot, with each element URI written out fully. This way of writing facts is similar to the Clojure based notation, with main change that the Clojure notation does use prefixes for URI abbreviation. This simplicity contrasts with the flexibility of the Turtle format, which can be more terse, but more complex to parse and generate. This also shows that a separate N-Triples</a> converter is not really needed. As long as the original knowledge bases does not use any named-graphs the result will be the same as with N-Triples.</p>
The final format that we aim to convert to is JSON-LD</a>. This is a format based on the JavaScript Object Notation JSON</a>, which allows for very easy interoperability with JSON based tools. </p>
Converting can be done with the following invocation:</p>
(</span>conv/convert-to-json-ld</span> fox-and-stork-named-graph-edn)
</span></code></pre>
resulting in the following JSON representation:</p>
{
</span>  "</span>@graph</span>" : [ {
</span>    "</span>@graph</span>" : [ {
</span>      "</span>@id</span>" : "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/fox</span>",
</span>      "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/gives-invitation</span>" : {
</span>        "</span>@id</span>" : "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation1</span>"
</span>      }
</span>    }, {
</span>      "</span>@id</span>" : "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation1</span>",
</span>      "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/has-food</span>" : {
</span>        "</span>@id</span>" : "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/soup</span>"
</span>      },
</span>      "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/has-invited</span>" : {
</span>        "</span>@id</span>" : "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/stork</span>"
</span>      },
</span>      "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/serves-using</span>" : {
</span>        "</span>@id</span>" : "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/shallow-plate</span>"
</span>      }
</span>    } ],
</span>    "</span>@id</span>" : "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner1</span>",
</span>    "</span>http://www.w3.org/2006/time#before</span>" : {
</span>      "</span>@id</span>" : "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner2</span>"
</span>    }
</span>  }, {
</span>    "</span>@graph</span>" : [ {
</span>      "</span>@id</span>" : "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation1</span>",
</span>      "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/serves-using</span>" : {
</span>        "</span>@id</span>" : "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/narrow-mouthed-jug</span>"
</span>      }
</span>    }, {
</span>      "</span>@id</span>" : "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation2</span>",
</span>      "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/has-food</span>" : {
</span>        "</span>@id</span>" : "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/crumbled-food</span>"
</span>      },
</span>      "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/has-invited</span>" : {
</span>        "</span>@id</span>" : "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/fox</span>"
</span>      },
</span>      "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/serves-using</span>" : {
</span>        "</span>@id</span>" : "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/narrow-mouthed-jug</span>"
</span>      }
</span>    }, {
</span>      "</span>@id</span>" : "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/stork</span>",
</span>      "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/gives-invitation</span>" : {
</span>        "</span>@id</span>" : "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation2</span>"
</span>      }
</span>    } ],
</span>    "</span>@id</span>" : "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner2</span>"
</span>  }, {
</span>    "</span>@id</span>" : "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/fox</span>",
</span>    "</span>@type</span>" : "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/animal</span>"
</span>  }, {
</span>    "</span>@id</span>" : "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/stork</span>",
</span>    "</span>@type</span>" : "</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/animal</span>"
</span>  } ]
</span>}
</span></code></pre>
The biggest benefit of this format is the compatibility with JSON based tools and techniques. Regular JSON parsers, encoders and other tooling will just work, giving the format a very wide reach. Similarly to this our Clojure based approach uses EDN</a> as its basis. This is a subset of Clojure, notably its notation of data values, and is used by Datomic</a> and others as a data transfer format.</p>
To summarize, we have seen how converting Linked Data from the Clojure representation to various other formats using the Aesopica</a> library is just a function invocation away. We have also looked at some of the differences between various syntaxes, notably the benefits that they provide: Turtle/TriG offers a lot of flexibity and shorthands for reading and writing, N-Quads simplicity of notation, and JSON-LD compatibility with an existing and well used standard. The Clojure representation is aimed creating a new, and hopefully interesting blend. It makes use of prefixes for easy reading and writing by human users, similarly to what is possible in Turtle. It has the simplicity of fact representation as triples and quads, like in N-Quads. Finally it uses a common, albeit not nearly as widespread, standard  a basis so it can make use of EDN based tooling. </p>
One interesting element, that the Turtle and Trig formats provide, is various short-hands for reading and writing. We believe this is a very useful feature, but of course the trade-offs of the shorthands versus the simplicity of notation must be taken into account. The format of which such shorthand will take shape, is therefor the topic for another article.</p>
Note that previous articles in this series can be also be found on this site: </p>

Part 1, General Introduction</a> covers the basic elements of Linked Data/RDF along with their representation in Clojure. It also introduces "The Fox and the Stork" formalised using Linked Data.</li>
Part 2, Literal Values</a> describes how literal values can be represented.</li>
Part 3, Named Graphs</a> describes the notion of representing, and naming, graphs, which allows for representing information about facts and graphs themselves.</li>
</ul>
As always, the functionality detailed in these articles can be found in the Aesopica</a> library for using Clojure to write Linked Data.</p>


Aesopica, Part 3: Named Graphs
2018-12-04T00:00:00+00:00
This article is the third part of a series, examining the use of the Clojure language for representing Linked Data, with examples from Aesop's stories. In part one</a> the basic elements of "The Fox and the Stork" story were formalised as Linked Data in Clojure, while in part two</a> we investigated how various literal values can be described. In this article we examine how information about facts themselves, such as meta-information, can be described with Linked Data. As always, the functionality detailed in these articles can be found in the Aesopica</a> library for using Clojure to write Linked Data.</p>
In Linked Data, facts are represented as triples of subjects, predicates and objects. For example, when representing the story of the "The Fox and the Stork" one fact that we want to represent is "The Fox gives an invitation." In this fact "The Fox" is the subject, the "gives an" is the predicate and "an invitation" is the object. Of course, as we mentioned in our previous articles, one of the strengths of Linked Data is that the elements are more precisely defined than just their natural language representations in a sentence. A Uniform Resource Identifier (URI</a>) is used to more formally identify these elements. This would make the previous fact to be written as follows, using the Turtle notation of RDF:</p>
http:</span>//www.newresalhaider.com/ontologies/aesop/foxstork/fox </span>http:</span>//www.newresalhaider.com/ontologies/aesop/foxstork/gives-invitation </span>http:</span>//www.newresalhaider.com/ontologies/aesop/foxstork/invitation1.
</span></code></pre>
When making using a base prefix for http://www.newresalhaider.com/ontologies/aesop/foxstork/fox</code> this could be shortened as: </p>
@base </span><http://www.newresalhaider.com/ontologies/aesop/foxstork/></span> .
</span><fox> <gives-invitation> <invitation1></span>.
</span></code></pre>
Using our Clojure based notation, that was introduced in the previous articles, we could write this same fact as follows:</p>
  {</span>::aes/context
</span>   {</span>nil </span>"</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/</span>"}
</span>   </span>::aes/facts
</span>   #{[</span>:fox :gives-invitation :invitation1</span>]
</span>     }}
</span></code></pre>
The above-mentioned fact is just one out of many needed to represent the full story of "The Fox and the Stork". In most cases a multitude of facts is required to represent the required knowledge. A set of facts, each consisting of subjects, predicates and objects, form a knowledge graph which provides us with a very general, but precise, way to represent knowledge.</p>
However there are scenarios when we want to represent knowledge about the facts themselves. One way Linked Data/RDF facilitates this is the use of the "named graphs". Named graphs allows us to associate an identifier (a [URI](Uniform Resource Identifier)) with a fact, or a set of facts. This essentially gives a name to a graph in the knowledge base, hence the notion of "named graphs". Such an identifier can then be used as a way to add information about the facts with which it is associated.</p>
The NQUADS</a> syntax for RDF illustrates one way such named graphs can be represented. In this representation all the elements of the fact: the subject, predicate, object and optionally a graph-name, are written out fully, separated by spaces and concluding with a dot (.</code>) . </p>
To take a single fact as an example, here follows a NQUADS format representation that details that "for the first invitation the Stork has been invited", and this fact is part of the "first dinner" named graph: </p>
<http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation1> <http://www.newresalhaider.com/ontologies/aesop/foxstork/has-invited> <http://www.newresalhaider.com/ontologies/aesop/foxstork/stork> <http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner1></span> .
</span></code></pre>
For this fact there are four elements to be represented, hence we can refer to these elements together as a quad, versus the notion of a triple for facts just consisting of a subject, predicate and object. As mentioned previously, URIs are used to precisely identify each element:  <http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation1></code> is the subject, <http://www.newresalhaider.com/ontologies/aesop/foxstork/has-invited></code> is the predicate and <http://www.newresalhaider.com/ontologies/aesop/foxstork/stork></code> is the object respectively. In addition the graph is identified by the URI <http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner1></code>. </p>
The big benefit of using such identifiers as names for the graphs is that they themselves can be part of facts. For example if we want to express that the facts contained inside the "first dinner" graph occur before the facts of the "second dinner" graph, we can use the fact: </p>
<http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner1> <http://www.w3.org/2006/time#before> <http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner2></span> .
</span></code></pre>
Note that this fact itself is not part of any named graph. In a knowledge base of facts this would make it a part of the "default graph". A default graph is a graph without any particular name. This makes the mixing of "regular" facts, where each fact consists of a triple, and facts in explicit named graphs, where each fact is a quad, possible in a single knowledge base.</p>
An example of a sightly expanded version using of "The Fox and the Stork"" story using named graphs in the NQUADS format can be therefor be as follows:</p>
<http://www.newresalhaider.com/ontologies/aesop/foxstork/fox> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://www.newresalhaider.com/ontologies/aesop/foxstork/animal></span> .
</span><http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner1> <http://www.w3.org/2006/time#before> <http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner2></span> .
</span><http://www.newresalhaider.com/ontologies/aesop/foxstork/stork> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://www.newresalhaider.com/ontologies/aesop/foxstork/animal></span> .
</span><http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation2> <http://www.newresalhaider.com/ontologies/aesop/foxstork/serves-using> <http://www.newresalhaider.com/ontologies/aesop/foxstork/narrow-mouthed-jug> <http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner2></span> .
</span><http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation2> <http://www.newresalhaider.com/ontologies/aesop/foxstork/has-food> <http://www.newresalhaider.com/ontologies/aesop/foxstork/crumbled-food> <http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner2></span> .
</span><http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation2> <http://www.newresalhaider.com/ontologies/aesop/foxstork/has-invited> <http://www.newresalhaider.com/ontologies/aesop/foxstork/fox> <http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner2></span> .
</span><http://www.newresalhaider.com/ontologies/aesop/foxstork/stork> <http://www.newresalhaider.com/ontologies/aesop/foxstork/gives-invitation> <http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation2> <http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner2></span> .
</span><http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation1> <http://www.newresalhaider.com/ontologies/aesop/foxstork/serves-using> <http://www.newresalhaider.com/ontologies/aesop/foxstork/narrow-mouthed-jug> <http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner2></span> .
</span><http://www.newresalhaider.com/ontologies/aesop/foxstork/fox> <http://www.newresalhaider.com/ontologies/aesop/foxstork/gives-invitation> <http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation1> <http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner1></span> .
</span><http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation1> <http://www.newresalhaider.com/ontologies/aesop/foxstork/has-food> <http://www.newresalhaider.com/ontologies/aesop/foxstork/soup> <http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner1></span> .
</span><http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation1> <http://www.newresalhaider.com/ontologies/aesop/foxstork/serves-using> <http://www.newresalhaider.com/ontologies/aesop/foxstork/shallow-plate> <http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner1></span> .
</span><http://www.newresalhaider.com/ontologies/aesop/foxstork/invitation1> <http://www.newresalhaider.com/ontologies/aesop/foxstork/has-invited> <http://www.newresalhaider.com/ontologies/aesop/foxstork/stork> <http://www.newresalhaider.com/ontologies/aesop/foxstork/dinner1></span> .
</span></code></pre>
Now that we introduced the concept of "named graphs" we now want introduce a way to represent them in the Clojure representation of Linked Data. Similarly on how in NQUADS the triples are extended to quads to indicate the name of the graph, we extend our previously introduced Clojure syntax</a> to be able to use quads for facts, as opposed to just triples. Similarly to NQUADS the, optional, fourth element of each fact represents the named graph identifier. Any regular triple based fact is part of the default graph in the knowledge base, similarly to the NQUAD representation.</p>
The resulting Clojure representation of above-mentioned Linked Data story can be written as follows:</p>
(</span>def </span>fox-and-stork-named-graph-edn
</span>  {</span>::aes/context
</span>   {</span>nil </span>"</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/</span>"
</span>    </span>:rdf </span>"</span>http://www.w3.org/1999/02/22-rdf-syntax-ns#</span>"
</span>    </span>:time </span>"</span>http://www.w3.org/2006/time#</span>"}
</span>   </span>::aes/facts
</span>   #{[</span>:fox :rdf/type :animal</span>]
</span>     [</span>:stork :rdf/type :animal</span>]
</span>     [</span>:fox :gives-invitation :invitation1 :dinner1</span>]
</span>     [</span>:invitation1 :has-invited :stork :dinner1</span>]
</span>     [</span>:invitation1 :has-food :soup :dinner1</span>]
</span>     [</span>:invitation1 :serves-using :shallow-plate :dinner1</span>]
</span>     [</span>:stork :gives-invitation :invitation2 :dinner2</span>]
</span>     [</span>:invitation2 :has-invited :fox :dinner2</span>]
</span>     [</span>:invitation2 :has-food :crumbled-food :dinner2</span>]
</span>     [</span>:invitation2 :serves-using :narrow-mouthed-jug :dinner2</span>]
</span>     [</span>:invitation1 :serves-using :narrow-mouthed-jug :dinner2</span>]
</span>     [</span>:dinner1 :time/before :dinner2</span>]}})
</span></code></pre>
The main difference between the Clojure representation and NQUADS is that the Clojure representation uses prefixes and NQUADS uses full URIs written out each time. This is a deliberate design choice in syntax from both perspectives. In NQUADS this allows the format to represent each fact on a single line, without the need for a lookup based on context for the full URI of elements. In the Clojure representation the prefixes allow for a much more compact fact representation that makes for easier reading and writing by human users. </p>
There are a number of other formats for writing Linked Data, some of which support named graphs. TriG</a> for example is an extension of the Turtle format used in previous articles in this series. JSON-LD</a> is also a very commonly used format for Linked Data that also supports named graphs. With the introduction of the Clojure way of writing Linked Data in this series, it makes sense to enable translating Linked Data into these formats for compatibility and reaching a wider audience. The facts on how to achieve this will be detailed in another article.</p>


Aesopica, Part 2: Literal Values
2018-09-14T00:00:00+00:00
This article is the second part of a series, examining the use of the Clojure language for representing Linked Data, with examples from Aesop's stories. Part one can be found on this site</a> where the basic elements of the Fox and the Stork story were formalised. In this article we examine how literal values can be represented, using Clojure, in Linked Data. The code to enable the functionality described in these articles can be found in the Aesopica</a> library for using Clojure to write Linked Data.</p>
As mentioned in the previous article the story of the Fox and the Stork is about the fox who invited the stork for a dinner. At the dinner soup was served from a shallow plate that the fox could eat but the stork could not. In return, the stork invited the fox to a dinner, where the food was served in a narrow mouthed jug. This time the fox could not reach the food, while the stork ate it happily. </p>
Elements of this story can be represented as Linked Data</a>, and in particular the Resource Description Framework (RDF)</a> that allows for a precise retelling of the story that is understandable to both humans and machines alike. </p>
For example, the part of the Linked Data we generated in the previous article is as follows:</p>
@base </span><http://www.newresalhaider.com/ontologies/aesop/foxstork/></span> .
</span>@prefix </span>rdf: </span><http://www.w3.org/1999/02/22-rdf-syntax-ns#></span> .
</span>
</span><fox> </span>rdf:</span>type </span><animal></span>.
</span><stork> </span>rdf:</span>type </span><animal></span>.
</span><fox> <gives-invitation> <invitation1></span>.
</span><invitation1> <has-invited> <stork></span>.
</span><invitation1> <has-food> <soup></span>.
</span></code></pre>
which details the elements of the story that the fox invites the stork, where soup is served. Elements that might be implicitly obvious to a human reader, but not to a program, that the fox and the stork are animals, are also represented in this fragment. These elements are describes as a set of facts, where each fact is a triple in the form of a subject, predicate and object. Each part of these facts in this example are represented as a Uniform Resource Identifier (URI)</a>, which are shortened with prefixes (i.e. "rdf"</code>) or the base URI (i.e. <fox></code> is a shorthand for http://www.newresalhaider.com/ontologies/aesop/foxstork/fox</code> ).</p>
Now suppose we want to expand on the elements of this story. For example, we want to give the fox and the stork a name, an age, describe their personalities, give a time for the dinners, etc. For many of these elements we want to simply use value as objects in the representations. For example the number 2</code> as a representative of the age of the fox. In such scenarios we do not use URIs in the facts but Literals</a>.</p>
Literals can be used to denote numbers, strings, dates and other such elements. In the Linked Data representation below we describe various attributes of the fox, the stork and the dinner with such literals.</p>
@base </span><http://www.newresalhaider.com/ontologies/aesop/foxstork/></span> .
</span>  </span>@prefix </span>rdf: </span><http://www.w3.org/1999/02/22-rdf-syntax-ns#></span> .
</span>  </span>@prefix </span>foaf: </span><http://xmlns.com/foaf/0.1/></span> .
</span>  </span>@prefix </span>xsd: </span><http://www.w3.org/2001/XMLSchema#></span>.
</span>
</span>  </span><fox> </span>rdf:</span>type </span><animal></span>.
</span>  </span><fox> </span>foaf:</span>name </span>"</span>vo</span>".
</span>  </span><fox> </span>foaf:</span>age </span>2.
</span>  </span><fox> <is-cunning></span> true.
</span>  </span><stork> </span>rdf:</span>type </span><animal></span>.
</span>  </span><stork> </span>foaf:</span>name </span>"</span>ooi</span>".
</span>  </span><stork> </span>foaf:</span>age </span>13.
</span>  </span><stork> <is-cunning></span> true.
</span>  </span><dinner1> <has-date> </span>"</span>2006-06-30T20:00:00</span>"^^</span>xsd:</span>dateTime
</span></code></pre>
As one can see in this example, representing literals is very similar to other objects. For example "vo"</code> in the triple <fox> foaf:name "vo"</code> represents the name of the fox. Note that the base of foaf</code> in foaf:name</code> and foaf:age</code> refers to the 'Friend of a Friend' ontology, that allows us to use the common terminology of this ontology to describe facts about the fox and the stork.  Literals such as 2</code> or true</code> describe the age, and whether the fox is cunning, respectively. These are called the lexical forms of the literals and while they also have explicit types (e.g. http://www.w3.org/2001/XMLSchema#string</code> or simply xsd:string</code> when using prefixes ), these types of literals are so common that writing the types explicitly is not required. </p>
The slightly more complicated case is the definition of the time of the dinner shown by "2006-06-30T20:00:00"^^xsd:dateTime</code> that shows off custom types for literals, or when we would like to give the type explicitly. Here the addition of the ^^xsd:dateTime</code> is an URI (with a prefix) describing how lexical form, i.e. "2006-06-30T20:00:00" exactly maps to a particular value. This allows for easier interpretation of such literal values for machines.</p>
As in the previous article, we aim to use the data representation and manipulation capabilities of Clojure to represent the above-mentioned fragment. Again, for the basic cases, such as strings, numbers, etc, we can be pretty straightforward and only use the lexical form, i.e. "vo"</code>, 2</code> or true</code> in or representations. For the cases where we also want to specify a custom datatype, we use a map such as:</p>
{</span>::aes/value </span>"</span>2006-06-30T20:00:00</span>" </span>::aes/type :xsd/dateTime</span>}
</span></code></pre>
where the keys ::aes/value</code> and ::aes/type</code> are representing the lexical form and datatype respectively. Note that aes</code> in these keywords, and other refers, to the core namespace of the Aesopica</a> library implementing this data representation. To full Clojure version of this example can be found below:</p>
  {</span>::aes/context
</span>   {</span>nil </span>"</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/</span>"
</span>    </span>:rdf </span>"</span>http://www.w3.org/1999/02/22-rdf-syntax-ns#</span>"
</span>    </span>:foaf </span>"</span>http://xmlns.com/foaf/0.1/</span>"
</span>    </span>:xsd </span>"</span>http://www.w3.org/2001/XMLSchema#</span>"}
</span>   </span>::aes/facts
</span>   #{[</span>:fox :rdf/type :animal</span>]
</span>     [</span>:fox :foaf/name </span>"</span>vo</span>"]
</span>     [</span>:fox :foaf/age 2</span>]
</span>     [</span>:fox :is-cunning true</span>]
</span>     [</span>:fox :has-weight 6.8</span>]
</span>     [</span>:stork :rdf/type :animal</span>]
</span>     [</span>:stork :foaf/name </span>"</span>ooi</span>"]
</span>     [</span>:stork :foaf/age 13</span>]
</span>     [</span>:stork :is-cunning true</span>]
</span>     [</span>:dinner1 :has-date </span>{</span>::aes/value </span>"</span>2006-06-30T20:00:00</span>" </span>::aes/type :xsd/dateTime</span>}]}}
</span></code></pre>
Of course given that this story is centuries old, it is unlikely that the dinner took place at 2006-06-30T20:00:00</code>. As always care must be taken when taking things literally.</p>


Aesopica, Part 1: General Introduction
2018-09-10T00:00:00+00:00
The stories called Aesop's Fables or the Aesopica, are an ancient collection of stories that have been passed down to modern day. These stories are of diverse origins they cover a wide variety of themes. Although originally intended for an adult audience, in later times were often used for the education of children.</p>
One of such stories is the tale of the Fox and the Stork. There are many versions of this fable, but the overall outline is generally as follows:</p>

"The fox invited the stork to dinner. At the dinner soup was served from a shallow plate, that the fox could eat but the hungry stork could not even taste. In turn the stork invited the fox to a dinner. Dinner was served in a narrow mouthed jug filled with crumbled food. This time the fox could not reach the food, while the stork ate."</p>
</blockquote>

    
    
    
        
        1884 fountain design depicting the story of the Fox and the Stork by Catalan sculptor Eduard Batiste Alentorn in Barcelona</h4>
        
        
    </figcaption>
</figure>

The intention of stories such as these, as well as text in general, is to convey meaning. However, in addition to humans, a new audience for text has come to light in recent years: machines. To facilitate this new audience a set of technologies has been developed to convey the meaning of text in a precise and unambiguous way that is easily understandable for both humans and machines alike. Many of these new methods fall under the umbrella of the Semantic Web</a>. The goal of the Semantic Web is to create a web of data where the meaning of the information is both human and machine understandable.</p>
One of the cornerstone technologies in conveying information for this purpose is Linked Data</a>, and in particular the Resource Description Framework (RDF)</a> standard that defines how this Linked Data can be expressed. I have written a short introduction to Linked Data</a> before but to summarize: it allows for the expressing information as a set of facts. These facts have the form of subject, predicate, object triples. A set of these facts is often called a knowledge base, or in an alternative view this can also been seen as a knowledge graph where the facts define the nodes and edges.</p>
In a Linked Data representation the story of Fox and the Stork would look something like this: </p>
@base </span><http://www.newresalhaider.com/ontologies/aesop/foxstork/></span> .
</span>@prefix </span>rdf: </span><http://www.w3.org/1999/02/22-rdf-syntax-ns#></span> .
</span>
</span><fox> </span>rdf:</span>type </span><animal></span>.
</span><stork> </span>rdf:</span>type </span><animal></span>.
</span><fox> <gives-invitation> <invitation1></span>.
</span><invitation1> <has-invited> <stork></span>.
</span><invitation1> <has-food> <soup></span>.
</span><invitation1> <serves-using> <shallow-plate></span>.
</span><stork> <gives-invitation> <invitation2></span>.
</span><invitation2> <has-invited> <fox></span>.
</span><invitation2> <has-food> <crumbled-food></span>.
</span><invitation2> <serves-using> <narrow-mouthed-jug></span>.
</span><fox> <can-eat-food-served-using> <shallow-plate></span>.
</span><fox> <can-not-eat-food-served-using> <narrow-mouthed-jug></span>.
</span><stork> <can-eat-food-served-using> <narrow-mouthed-jug></span>.
</span><stork> <can-not-eat-food-served-using> <shallow-plate></span>.
</span></code></pre>
This is in the Turtle syntax</a> of RDF. There are other types of syntax are available to represent Linked Data, for example in JSON form as JSON-LD</a>.</p>
To summarize a bit of what this Linked Data format does in this scenario, is that it uses Uniform Resource Identifiers (URIs)</a> to define the subjects, predicates and objects of each fact. This allows to precisely and unambiguously define and link the meaning between these elements. For example, the fact that the fox is a type of animal could be expressed by the triple with the full URIs: http://www.newresalhaider.com/ontologies/aesop/foxstork/fox http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://www.newresalhaider.com/ontologies/aesop/foxstork/animal .</code> Due to the fact that writing the full URIs can be quite cumbersome, the Turtle syntax uses two kinds of shorthands to help out. In this case one can define a base URI for the current document, http://www.newresalhaider.com/ontologies/aesop/foxstork</a>, as well as prefixes for other namespeaces, such as http://www.w3.org/1999/02/22-rdf-syntax-ns#</a>, with which the writing of each fact that would begin with these URI fragments could be shortened.</p>
When everything put together this format still describes the original story, albeit restructured into separate facts.</p>
There exists many tools for handling Linked Data such as the above story. For example APIs, such as Jena</a>, can aid in the creation, storage and querying of data made available in such a fashion. Of course more and better tools and techniques are always welcome. In this article in particular we hope to describe how we can use the Clojure programming language to enable working with Linked Data. </p>
Clojure</a> is a language that offers a lot of benefits. The focus on manipulating pure data, with immutable data-structures and functional programming, provides an excellent way to organize code. The ability to inter-operate with the Java and JavaScript ecosystems, allows for the use of many mature libraries as well as many avenues for deployment. </p>
To use the data manipulation capabilities of Clojure to enable the Semantic Web, seems like a natural combination. Some previous works also aimed at exploring this area, notably EDN-LD</a> which gives a convention and a library for working with Linked Data.</p>
In this article we will also explore how we can use Clojure to interact with Linked Data. In our case we will focus on the creation Linked Data from a Clojure environment and we might take different conventions compared to previous work, so we start with a fresh implementation. </p>
In Clojure, information is directly represented as data, as opposed to it being encapsulated into various other abstractions such as objects. A large subset of elements data in Clojure is also a data format called the Extensible Data Notation (EDN)</a>. The built-in elements in this notation are nil, booleans, strings, characters, symbols, keywords, integers, floating-point numbers, lists, vectors, maps and sets. The meaning behind most of these elements is relatively straightforward, so we only give a brief summary of them here and some examples. </p>
Nil</h5>
An empty or non-existent element is represented by nil</code>.</p>
Booleans</h5>
A boolean value can be true</code> or false</code>. </p>
Strings</h5>
Strings are written in double quotes, for example: "This sentence is a string."</code>.</p>
Characters</h5>
Characters representing single characters, and are preceded by a backslash, for example \c</code> or \newline</code>. </p>
Symbols</h5>
Symbols are representing identifiers, written by a set of characters (with a few additional rules). Examples of identifiers are for example foo</code>, clojure.core</code>, clojure.string/split</code>. As some of these examples show, in Clojure they are used, among other things to refer to modules and functions. Another interesting feature, as the clojure.string/split</code> example shows, is that they can be namespaced which helps to organize symbols and avoid name collisions.</p>
Keywords</h5>
Keywords are very similar to symbols but they are identifiers that refer to themselves. They are constructed much like symbols, but with a leading :</code>. Examples of keywords are :fruit</code> or :company.persons/name</code>. </p>
Integers and Floats</h5>
Integers and floats (floating point numbers) are used, as expected, to write numbers 3</code> or 4.5</code> for example. </p>
All these elements described above can be put in collections. </p>
Lists</h5>
Lists are a sequence of values enclosed in ()</code>, for example (2 "A string." false)</code>. </p>
Vectors</h5>
Vectors are a sequence of values enclosed in []</code>, for example [true nil :company/name]</code>. which are designed for random access of its elements. </p>
Sets</h5>
Sets are collections of unique values enclosed in #{}</code>, such as #{:fruit 2}</code>. </p>
Maps</h5>
Finally maps are key value pairs, enclosed in curly braces {}</code>, for example {:name "John Smith", :age 4}</code>, where each key is unique. Of course collections can also nested any type of collection.</p>
Using this notation elements of EDN, we can build an EDN based version of the story of the Fox and the Stork, using some conventions. </p>
Given that in many practical cases we are probably going to shorten URIs with prefixes when writing, we can use a keyword for denoting elements. In the case where we would use the base prefix, we can just use a regular, non-namespaced, keyword, i.e. :fox</code>, and in cases where we would refer to any other prefix we can use namespaced keywords, i.e. :rdf/type</code>. A full fact could then be described with a relatively straightforward vector, for example [:fox :rdf/type :animal]</code> and the knowledge base with a set of facts such as #{[:fox :rdf/type :animal] [:stork :rdf/type :animal]}</code>. </p>
Of course this means that in addition to facts we also need some data for the context, in which we store the base and other prefixes and to what they map to, to be able to fully build an equivalent Linked Data representation. The context will be a map of the relevant prefixes as keys, as well as nil for the base prefix. For the above example this means that the below example will describe the context needed to resolve all the full URIs: </p>
{</span>nil </span>"</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/</span>"
</span>:rdf </span>"</span>http://www.w3.org/1999/02/22-rdf-syntax-ns#</span>"}
</span></code></pre>
Putting everything together, to have a full Linked Data graph we need a context and a set of facts, so the overall structure will be a map where these are both defined:</p>
  {</span>::aes/context
</span>   {</span>nil </span>"</span>http://www.newresalhaider.com/ontologies/aesop/foxstork/</span>"
</span>    </span>:rdf </span>"</span>http://www.w3.org/1999/02/22-rdf-syntax-ns#</span>"}
</span>   </span>::aes/facts
</span>   #{[</span>:fox :rdf/type :animal</span>]
</span>     [</span>:stork :rdf/type :animal</span>]
</span>     [</span>:fox :gives-invitation :invitation1</span>]
</span>     [</span>:invitation1 :has-invited :stork</span>]
</span>     [</span>:invitation1 :has-food :soup</span>]
</span>     [</span>:invitation1 :serves-using :shallow-plate</span>]
</span>     [</span>:stork :gives-invitation :invitation2</span>]
</span>     [</span>:invitation2 :has-invited :stork</span>]
</span>     [</span>:invitation2 :has-food :crumbled-food</span>]
</span>     [</span>:invitation2 :serves-using :narrow-mouthed-jug</span>]
</span>     [</span>:fox :can-eat-food-served-using :shallow-plate</span>]
</span>     [</span>:fox :can-not-eat-food-served-suing :narrow-mouthed-jug</span>]
</span>     [</span>:stork :can-eat-food-served-using :narrow-mouthed-jug</span>]
</span>     [</span>:stork :can-not-eat-food-served-suing :shallow-plate</span>]}}
</span></code></pre>
I have started a small library for manipulating Linked Data structures written this way, with the name Aesopica</a>. It is in very early stages, where the current main functionality is to translate Linked Data written this way into the Turtle format described above.</p>
Of course there are lot of other elements of Linked Data that needs to be represented in this that we did not tackle yet. In addition there are also a large number of Clojure libraries that could be used to make writing and using Linked Data in this fashion easier. How these features could be achieved however is a story for another time.</p>


Excavating a Common Treasure
2018-06-30T00:00:00+00:00
History is full of legends of ancient treasures and powerful artifacts, lost to time or hidden by purpose, that would bring glory, power and riches to their discoverer.</p>

    
    
    
        
        A winged bull depicted on an on archaeological artifact from the Assyrian empire between 1400 and 1200 BC. Cylinder Seal with Winged Bull - Walters Art Museum Licensed under CC0.</h4>
        
        
    </figcaption>
</figure>

Although the history of software development is much shorter than the history of mankind, there already exists artifacts that have near mythological status when it comes to software development, yet often go hidden underneath the surface. One of these is the language of Common Lisp.</p>
Common Lisp is a language originally from the early 80s, and was a standardization of many dialects of the Lisp programming language, which explains both the Common and Lisp part of the name. Lisp itself is one of the oldest general purpose programming languages, first specified in 1958. From Lisp many ideas that we consider common in other languages were popularized, such as conditionals (the if and else</code> structure), notion of functions as first class elements, and many others (see this article</a> on more of these ideas). Common Lisp is also often mentioned</a> as an amazing and fun language to learn, which makes it the clear-cut choice for this article.</p>
As the story goes it gives the user untold levels of productivity and joy of development. One other big advantage is that Common Lisp used to be the language of A.I. research. There is lots of existing code out there for systems on the subject of knowledge representation, reasoning and planning. Given the resurgence of fields such as a Semantic Web</a> that make use of such A.I. systems, learning from such tools and techniques, or even using them directly, can be invaluable. In this post, I aim to find out a very small portion of the riches we can dig up using Common Lisp, and to share this experience with the reader. </p>

    
    
    
        
        Cylinder Seals were common objects that were used to designate ownership in ancient Mesopotamia. Although describing to a different notion of 'common' these show remarkable of craftsmanship that we hope to introduce ourselves to in Common Lisp. Cylinder Seal with Two Heroes and a Tree - Walters Art Museum Licensed under CC0.</h4>
        
        
    </figcaption>
</figure>

To add as a disclamer, I do have experience with a similar language, Clojure</a> that I use in my day-to-day programming, as well as with Emacs</a> (namely Spacemacs</a>), so I probably have some head start in using and editing the language. Nonethess I aim to make this introduction accessible to readers without such knowledge. </p>
With any treasure hunt, preparation is often key. As we are unfamiliar with the language itself it is good to read guides and tutorials to prepare. The guide to selecting the right equipment for our adventure is the cl-cookbook</a>, which describes how to get started with Common Lisp.</p>
Common Lisp is a language with multiple implementations. As recommended by our guide we are using Steel Bank Common Lisp (SBCL)</a> as distribution and using Roswell</a> to manage it. Alternatively a more straightforward install is Portacle</a> that could be used.</p>
Next is testing whether our implementation is working. We run the Read–Eval–Print Loop (REPL), which in the case of Roswell can be done with the command ros run</code>. What the REPL allows us to do is interactively develop our code by reading our input, evaluating it and printing the results. This is one of the many features in programming that originated with Lisp, but has since made its way to many other languages. </p>
Now the question is how we should write Common Lisp. One really nice resource for quickly familiarizing with new languages is Learn X in Y minutes, which also has a nice introduction to Common Lisp</a>. One of the most striking features of a Lisp, is the syntax of it. Elements are either atoms, that evaluate to themselves, such as the number 4</code> or s-expressions which are a list of expressions in brackets, such as (+ 3 2)</code>. Atoms evaluate to themselves, so if we write 4</code> in the REPL, we get 4</code> returned back. For s-expressions they are in the form of (function param1 param2 ...)</code>, so in the case of (+ 3 2)</code> this will evaluate to 5</code>. One can of course nest these forms, for example (+ 3 (+ 3 2))</code> will result in 8</code>. In addition to functions there are macros available creating completely different forms to be translated into code. We will see some examples of this later in the article.</p>
With the REPL now working it is time to set up a project where we can write down all the code we need for our digging. According to our cookbook tutorial we can do this from the REPL, by getting cl-project</a>, and using it. </p>
(ql:quickload "</span>cl-project</span>")
</span>(cl-project:make-project </span>#P"./common-treasure"</span>)
</span></code></pre>
This will give us the outline of project in the common-treasure</code> directory. The files generated include a readme, systems for the project itself as well as tests and of course some skeletons for a test and a package itself respectively.</p>
First we are going to start off figuring out the package itself. Like in many other languages, functionality can be grouped together units, in this case packages. The generated file outline looks like this:</p>
(</span>defpackage</span> common-treasure
</span>  (:use :cl))
</span>(</span>in-package</span> :common-treasure)
</span>
</span>;; blah blah blah.
</span></code></pre>
From the looks of it, it says to us that it defines a package named common-treasure</code> using :cl</code>, which I assume is another package. There is also a call to in-package</code> which looks like is there to ensure that everything else that follows is also in this package. Finally the ;; blah blah blah.</code> portion looks like a comment, something to describe functionality with, but nothing to be executed.</p>
Now in order to see if everything is working correctly let's load this package. Instead of using the REPL from the command line, we are using it directly from our editor, in this case an Emacs</a> distribution named Spacemacs</a>. Without going into much detail about Emacs it can be described as a highly configurable editor for pretty much any programming language out there. SLIME: The Superior Lisp Interaction Mode for Emacs</a> ensures that our interaction with the REPL and the various language features go much more smoothly by integrating it with our editor of choice. </p>
Emacs is incredibly configurable, which is where Spacemacs comes in. It provides a set of curated configurations named layers, that make setting up the configuration for a particular language a breeze. We just enable the common-lisp layer</a> of Spacemacs in the configuration file, add a small piece to let Emacs know of our Common Lisp distribution (see link), and we are good to go.</p>

With everything setup now we can start our treasure hunt a bit more in earnest by starting SLIME. We are greeted by the message "Connected. Hack and be merry!" as well as a REPL in a window. Lets start to dig around by figuring out what exactly all the elements do. We can use the command slime-describe-symbol</code> to get some more information about defpackage</code> and what it does. This brings up some documentation on defpackage</code>. It shows us that this is a macro, as well as the various options one can use with it. A macro, without going into too much detail, is essentially a way to convert forms into different ones before evaluating them. We can get a bit more info on the workings by calling the command slime-hyperspec-lookup</code>. This takes us to the Common Lisp Hyperspec</a> with more detailed docs. The pages for defpackage</code></a> and inpackage</code></a> more or less confirm their use is as we expected.</p>
Now it is time to start writing some new code. We remove the comments and create a function named hello-treasure</code>, that should just print the text "Hello Treasure". We add the following to our file:</p>
(</span>defun </span>hello-treasure </span>()
</span>  (</span>print </span>"</span>Hello Treasure!</span>")
</span>  )
</span></code></pre>
in place of the comments, and we run the command slime-complile-and-load-file</code>. Compilation succeeds, but when we aim to call the function  (common-treasure:hello-treasure)</code> get an error when trying to call the function, saying "The symbol "HELLO TREASURE" is not external to the "COMMON TREASURE" package.". This due to the fact that we did not declare the function external in the defpackage</code> package declaration. </p>
What is really interesting is that, although we have this issue, instead of just failing the SLIME REPL presents us with a number of options. For example we can  0: [Continue] Use symbol anyway.</code>, 1: [Retry] Retry SLIME REPL evaluation request.</code>, 2: [*Abort] Return to SLIME's top level.</code>. In this case we can just abort, and modify the package declaration, but it is interesting to be offered all these options.</p>
By adding (:export :hello-treasure)</code> to the defpackage</code> macro, we can do another call and it indeed returns "Hello Treasure!" in the REPL.</p>
Now that we got the basics down we would like to have a slightly bigger example in which we make use of an existing library. While digging around for the software languages and libraries of the past is often the matter of some searching, reading, coding and a bit of trial and error. Digging around in the world is process governed by various regulations and laws. This is especially true in cities with long history, where a regular building site can easily unearth archaeological finds.</p>

    
    
    
        
        Digging in the certain urban areas can easily lead to excavations done before any subsequent work is done. Romano-Celtic temple excavated in London.</h4>
        
        
    </figcaption>
</figure>

In particular, there are often a set of regulations that specify whether one can dig in areas of archaeological interests. As an example we use a portion of the rules by the municipality of Utrecht, as described on a map</a> of Utrecht and surrounding areas (in Dutch).</p>
These regulations can be summarized by a number of rules, based on two conditions: the type of area and the size of the area that is to be disturbed. </p>


If the area is of high archeological value, then a permit is required no matter what the size of the area is to be disturbed.</p>
</li>

If the area is of high archeological expectation, then a permit is required when the size of the area that is to be disturbed is larger than 100m^{2</sup>.</p>
</li>}
If the area is of archeological expectation, then a permit is required when the size of the area that is to be disturbed is larger than 1000m^{2</sup>.</p>
</li>
</ul>}
Such regulations can be encoded into a regular function that returns a true or false to the question "Is the a permit required?" based on two parameters, the size of the area and the type of the area. However this could be more complicated and harder to maintain if the set of regulations, along with conditions and outcomes, become more complex.</p>
There are various approaches to deal with this issue. One of these is to encode the facts on which our answer relies into a knowledge base, and have a general mechanism to derive new knowledge from existing facts. This later process is often denoted as inferring or reasoning to derive new knowledge. Such an approach can make it easier to use and maintain such a system.</p>
For this article we are going to implement such an approach using LISA</a>. This is a production rule system, in which the knowledge base is encoded in a set of facts, and rules are uses as a way to derive new knowledge. It is easy to see how such a system would be a good fit for our problem: our domain restrictions are already formulated as a set of rules. It would also allow us to use some existing libraries, to see more of what Common Lisp has to offer.</p>
To start off, first we need to add a dependency to LISA into our system. One of the files created by cl-project is an asdf</a> file that describes how to build the software we are creating. Here we can add the dependency on Lisa, resulting in the following file:</p>
#|
</span>  This file is a part of common-treasure project.
</span>|#
</span>
</span>(defsystem "</span>common-treasure</span>"
</span>  :version "</span>0.1.0</span>"
</span>  :author ""
</span>  :license ""
</span>  :depends-on ("</span>lisa</span>") </span>;; Here we add the dependency on LISA.
</span>  :components ((:module "</span>src</span>"
</span>                :components
</span>                ((:file "</span>common-treasure</span>"))))
</span>  :description ""
</span>  :long-description
</span>  </span>#.</span>(</span>read</span>-file-</span>string
</span>     (subpathname *</span>load-pathname</span>* "</span>README.markdown</span>"))
</span>  :in-order-</span>to </span>((test-op (test-op "</span>common-treasure-test</span>"))))
</span></code></pre>
Now we just need to make sure the dependencies needed are actually downloaded and available. For this we use quicklisp</a> which is the library manager Common Lisp. By using the command (ql:quickload "common-treasure")</code> we can get all the dependencies for our system, which in this case consists of just LISA.</p>
Now in order to translate our example scenario, we look at some of the examples that are available for LISA. A typical system consists of a knowledge base, in which facts are defined as Common Lisp Object System (CLOS)</a> objects and rules to manipulate such instances as the reasoning method. </p>
CLOS is a way in which Common Lisp can do object-oriented programming, and LISA co-opts this as a way to represent facts in the knowledge base. Without going into detail on object-oriented programming theory, this allows one to, define classes, create instances of those classes and define methods that make use of those classes (see this nice intro on Common Lisp Object System (CLOS)</a> for its general use). </p>
In the context of LISA, these allow us to represent classes of facts, specific instances of facts, and methods that can be used with those facts. For example we can represent the type of facts in our domain with two classes of objects: one that represents "area" and one that whether "a permit is required". In Common Lisp these can be defined with the defclass</code> macro. So for our example the following would be a call for permission requirement:</p>
(</span>defclass</span> permit-required ()
</span>  ()
</span>)
</span></code></pre>
In case of the area, we need to represent two facts about it: what the archaeological type of it is, and what size of the area is disturbed. Such elements of the classes are described in CLOS using slots. There are many possible options that are available to describe a slot:</p>

:initarg</code> describes the name of the slot.</li>
:initform</code> is the default value it is given initially.</li>
:accessor</code> can be used to define a function for accessing the slot value.</li>
</ul>
Putting this together gives us the following class for representing the area:</p>
(</span>defclass</span> area ()
</span>  ((archeological-type :initarg :archeological-type :initform "" :accessor :archeological-type)
</span>   (size-disturbed :initarg :size-disturbed :initform </span>0.0</span> :accessor :size-disturbed)
</span>    ))
</span></code></pre>
Now that we figured out how represent facts, we need to encode the rules of our domain into our system. We can do this with 3 rules that describe what to do in the cases of high archeological value</code>, high archeological expectation</code> and archeological expectation</code> </p>
The rules themselves described with three parts:</p>

The antecedent, which is the set of conditions that have to be true. </li>
The arrow sign =></code> .</li>
The consequent, which contains the facts that need to be derived or any actions that we want to perform. </li>
</ul>
In the case of LISA, and in many other rule based definitions, the antecedent and the consequent are also called Left Hand Side (LHS) and Right Hand Side (RHS), as they are respectively on the left and right hand side of the arrow. What exactly can be put into the LHS and RHS is dependent on the language for the rule definitions, but one standard way they are used is in the LHS the conditions are described based on the facts that could be in the knowledge base, and on the RHS facts are added in the knowledge based on the meaning of the rule. For example for one of our rules, if the know the fact that our area is of high archaeological value (LHS), we can derive the fact that we need a permit (RHS). </p>
Describing our three rules this way in LISA looks as follows, using the defrule</code> macro:</p>
;; If the area is of high archeological value, then a permit is required no matter what the size of the area is to be disturbed.
</span>(defrule high-value () (area (archeological-type "</span>high-value</span>")) => (</span>assert </span>((</span>make-instance</span> 'permit-required))))
</span>
</span>;; If the area is of high archeological expectation, that a permit is required when the size of the area that is to be disturbed is larger than 100m<sup>2</sup>.
</span>(defrule high-expectation () (area (archeological-type "</span>high-expectation</span>") (> size-disturbed </span>100</span>))  => (</span>assert </span>((</span>make-instance</span> 'permit-required))))
</span>
</span>;; If the area if of archeological expectation, that a permit is required when the size of the area that is to be disturbed is larger than 1000m<sup>2</sup>.
</span>(defrule expectation () (area (archeological-type "</span>expectation</span>") (> size-disturbed </span>1000</span>))  => (</span>assert </span>((</span>make-instance</span> 'permit-required))))
</span></code></pre>
The rules are reasonably straightforward, in that they require the condition matching in the knowledge base, before they assert an instance of the permit-required</code> fact into the knowledge base. </p>
These together already describe the domain, but for this article we also show how things work with two examples. In one scenario we assert that we have an area with high-value</code> archaeological type and a to be disturbed are with the size of 2000m^{2</sup>. In this case we would need a permit, due to our high-value</code> rule. In the second scenario, the area is of high-expectation</code> but the size of the area disturbed is only 10m^{2</sup>. In this case none our rules will fire, and no permit-required</code> fact instance is put into the knowledge base.</p>}}
Executing the system is just a matter of adding these facts in the knowledge base and letting the system run. In LISA adding a fact can be done using assert</code> similarly to what the RHS of the rules are doing. To actually run the system, unsurprisingly, the function run</code> can be called. To see what facts are in the knowledge base the function facts</code> prints them out for us to see. To tie things together, we make sure that the knowledge base is reset before we execute anything using reset</code>. </p>
Below is the full example implementation that we put together:</p>
(</span>defpackage</span> common-treasure
</span>  )
</span>
</span>(</span>in-package</span> :lisa-user)
</span>
</span>(reset)
</span>
</span>(</span>defclass</span> area ()
</span>  ((archeological-type :initarg :archeological-type :initform "" :accessor :archeological-type)
</span>   (size-disturbed :initarg :size-disturbed :initform </span>0.0</span> :accessor :size-disturbed)
</span>    ))
</span>
</span>(</span>defclass</span> permit-required ()
</span>  ()
</span>)
</span>
</span>;; If the area is of high archeological value, then a permit is required no matter what the size of the area is to be disturbed.
</span>(defrule high-value () (area (archeological-type "</span>high-value</span>")) => (</span>assert </span>((</span>make-instance</span> 'permit-required))))
</span>
</span>;; If the area is of high archeological expectation, that a permit is required when the size of the area that is to be disturbed is larger than 100m<sup>2</sup>.
</span>(defrule high-expectation () (area (archeological-type "</span>high-expectation</span>") (> size-disturbed </span>100</span>))  => (</span>assert </span>((</span>make-instance</span> 'permit-required))))
</span>
</span>;; If the area if of archeological expectation, that a permit is required when the size of the area that is to be disturbed is larger than 1000m<sup>2</sup>.
</span>(defrule expectation () (area (archeological-type "</span>expectation</span>") (> size-disturbed </span>1000</span>))  => (</span>assert </span>((</span>make-instance</span> 'permit-required))))
</span>
</span>;;Test fact 1
</span>;; (assert ((make-instance 'area :archeological-type "high-value" :size-disturbed 2000)))
</span>
</span>;;Test fact 2
</span>(</span>assert </span>((</span>make-instance</span> 'area :archeological-type "</span>high-expectation</span>" :size-disturbed </span>10</span>)))
</span>
</span>(facts)
</span>
</span>(run)
</span>
</span>(facts)
</span></code></pre>
Note that here the fact 1</code> is commented out, to test the assert of fact 2</code>, so that running the other scenario is just a matter of commenting and un-commenting the relevant lines. </p>
For the first scenario, with fact 1, the output of our initial program is as follows:</p>
[package common-treasure]#<INITIAL-FACT </span>; id 0 {1002669853}>
</span>#<AREA </span>; id 1 {1002745373}>
</span>For a total of 2 facts.
</span>#<INITIAL-FACT </span>; id 0 {1002669853}>
</span>#<AREA </span>; id 1 {1002745373}>
</span>#<PERMIT-REQUIRED </span>; id 2 {100283A363}>
</span>For a total of 3 facts.
</span></code></pre>
Note that we print out our facts twice, once before the run and once after, but it shows that now the permit requirement was derived.</p>
For the case of using fact 2</code> , we get the following output:</p>
[package common-treasure].#<INITIAL-FACT </span>; id 0 {1004D40AB3}>
</span>#<AREA </span>; id 1 {10020107C3}>
</span>For a total of 2 facts.
</span>#<INITIAL-FACT </span>; id 0 {1004D40AB3}>
</span>#<AREA </span>; id 1 {10020107C3}>
</span>For a total of 2 facts.
</span></code></pre>
This shows that our fact base did not change due to our running the reasoning system.</p>
So there we have it, a bit of a peek of what can be done with Common Lisp. I have to say the language and its various features are quite a bit daunting, as I feel I have only scratched the surface of what is possible, with both interacting with the language as well as with A.I. tools such as LISA. That said digging into the language and seeing how it can handle a small, but also relevant scenario, felt like a worthwhile journey, and that is an experience always worth treasuring.</p>


Spec-stacular Spider-Man
2018-04-29T00:00:00+00:00
Spider-Man is one of the most iconic heroes of the Marvel universe. Created by Stan Lee and Steve Ditko, Spider-Man is a regular teenager named Peter Parker, who due to being bitten by a radio-active spider, gains abilities such as the proportional strength of a spider, wall crawling and a spider sense to detect upcoming danger. One of the biggest draws of Spider-Man that although he is a superhero and fought various villains from cosmic beings to petty criminals, he also had to deal with regular everyday problems, such as money issues, school life and the pressure of a job.</p>

    
    
    
        
        The Spectacular Spider-Man © Marvel Entertainment</h4>
        
        
    </figcaption>
</figure>

In software there are also everyday problems which one has to tackle before one can defeat the villains of the domain at hand. One of these everyday problems is the issue of data validation. Data validation is the process of ensuring that the elements of the data are correct. This process has to be done in pretty much all domains when working with actual data. Consider the financial domain where a financial product can only be made available if the right requirements are fulfilled in the request. If the request is not written correctly then the request needs to be denied. In the legal and regulatory domains certain information that is required for a law must be fulfilled, otherwise costly corrections or fines can follow. Another good example is the clinical domain, where the a patients data needs to be transferred to an application. Here is it essential that this data fulfills the requirement for requesting a clinical procedure or a medication, as any mistake can lead to huge negative impact on the health of the patient.</p>
One relatively recent tool that can be used to solve this problem is the clojure.spec</a> library in the Clojure programming language. In this article we aim to explain, alongside Spider-Man, how these specs can be used to tackle the data validation problem in a spectacular way. As this library relies on the Clojure language some knowledge of Clojure is needed. In order to make this article understandable to those without such prior expertise we introduce some aspects of Clojure. In particular we focus on two features of it: the way information (data) is represented and the fact that it is a Lisp.</p>
In Clojure data is represented with relatively few elements that are combined together. Take for example a scenario where we want to create a profile of Spider-Man, as taken from the Marvel wiki entry on Spider-Man</a>.</p>
The full name of Spider-Man can be represented in text form as a string. Like in many other languages the text is placed in between quotation marks.</p>
"</span>Peter Benjamin Parker</span>"
</span></code></pre>
For the numbers to represent his relative power in the Marvel universe, we use natural numbers (we leave the concepts and issues surrounding very large or floating point numbers out in this article). In case of Spider-Man his durability is 5:</p>
5
</span></code></pre>
Of course having just a value of the name and the durability of Spider-Man just floating around makes the representation somewhat incomplete, as they are not attached to the concepts of "name" or "durability". Just like how Spider-Man needs buildings to sling off of, we need a representation that links the values with what they represent. In Clojure, keywords are often used for this purpose. </p>
:real-name
</span></code></pre>
:spider-man-spec.core/name
</span></code></pre>
::durability
</span></code></pre>
Keywords are symbolic identifiers. Think of them as symbols, much like one would use a string, but with some special powers attached. They are text prefaced by ":", as it can be seen in the keyword :real-name</code>. They have the ability to be namespace-qualified, such as :spider-man-spec.core/name</code> which indicates that this is the keyword in the spider-man-spec.core</code> namespace. Namespaces are what allows us to modularize our data and code, by grouping them under together a single identifier. In our case this is spider-man-spec.core</code>. This namespacing ensures that our definition of the concepts of "name", "real-name", "durability", etc. can remain distinct from any other use of similar concepts. Finally, when writing internally to the library which uses the namespace, or when aliasing to it, we can just shorten the keyword with "::", such as in ::durability</code>. </p>
Keywords come with some nice implementation details, such as fast equality checks and some other powers we will show in the future. This makes them the preferred keys in data-structures such as maps. And speaking of maps they allow the description of information in key-value pairs, as written between curly braces in the small example below. </p>
  {</span>::name </span>"</span>Spider-Man</span>"
</span>   </span>::real-name </span>"</span>Peter Benjamin Parker</span>"}
</span></code></pre>
The curly backets around the pairs express the keys and values in a map in Clojure. In the above example ::name</code> and ::real-name</code> are the key and value pairs for "Spider-Man"</code> and "Peter Benjamin Parker"</code> respectively. </p>
Maps are just one of the ways one can describe a collection of elements. You also have sets, collections in which each element is unique. This can be done with a hashtag and some curly brackets "#{}". In the example below we list the current and former affiliations of Spider-Man.</p>
{</span>::current-affiliations </span>#{"</span>Avengers</span>"}
</span> </span>::former-affiliations </span>#{"</span>Secret Defenders</span>" "</span>New Fantastic Four</span>" "</span>The Outlaws</span>"}}
</span></code></pre>
Note that how sets are used within maps to represent this knowledge. This is actually a common way to represent knowledge in Clojure: you combine all the various data representations directly. This way you can have a list containing maps, with keywords as keys and values that contain maps and strings, where the maps contain numbers, etc. You have these data-structures in pretty much all commonly used programming languages. Where Clojure differs from many is that it does not put (almost any) sugaring or abstraction on top. </p>

    
    
    
        
        Spider-Man swinging around the city. © Marvel Studios</h4>
        
        
    </figcaption>
</figure>

Just as Spider-Man is often at his best when he is just being "plain old Spidey", having data represented this way has some nice advantages. The biggest is simplicity. Instead of learning to work with specific wrappers, objects, prototypes, etc on top of this data, that can differ between applications and libraries, it is enough to learn how to handle and manipulate maps, list, sets once. This knowledge can be then reused in any domain, and frees up the attention of the programmer to focus on the domain problem, and not the exact way the data was wrapped up in a library. </p>
This of course also means that a system, such as clojure.spec, that aims at data validation in Clojure, has to handle the above-mentioned style of composition well. But before we get ahead of ourselves lets finish up by providing the profile of Spider Man.</p>
(</span>def </span>spider-man-profile
</span>  {</span>::name </span>"</span>Spider-Man</span>"
</span>   </span>::real-name </span>"</span>Peter Benjamin Parker</span>"
</span>   </span>::identity ::secret
</span>   </span>::affiliations
</span>                   {</span>::current-affiliations </span>#{"</span>Avengers</span>"}
</span>                    </span>::former-affiliations </span>#{"</span>Secret Defenders</span>" "</span>New Fantastic Four</span>" "</span>The Outlaws</span>"}}
</span>   </span>::power-grid </span>{
</span>                 </span>::durability 3
</span>                 </span>::energy 4
</span>                 </span>::fighting 5
</span>                 </span>::intelligence 4
</span>                 </span>::speed 5
</span>                 </span>::strength 4
</span>                 }
</span>   }
</span>  )
</span>
</span>
</span>(</span>def </span>vulture-profile
</span>  {</span>::name </span>"</span>Vulture</span>"
</span>   </span>::real-name </span>"</span>Adrian Toomes</span>"
</span>   </span>::identity ::publicly-known
</span>   </span>::affiliations
</span>                   {</span>::current-affiliations </span>{}
</span>                    </span>::former-affiliations </span>#{"</span>Sinister Twelve</span>" "</span>Sinister Six</span>"}}
</span>   </span>::power-grid </span>{
</span>                 </span>::durability 4
</span>                 </span>::energy 3
</span>                 </span>::fighting 4
</span>                 </span>::intelligence 4
</span>                 </span>::speed 5
</span>                 </span>::strength 3
</span>                 }
</span>   }
</span>  )
</span>
</span>(</span>def </span>spider-man-characters </span>[spider-man-profile vulture-profile])
</span></code></pre>
Oh no, our Spider Senses should be tingling. It is Vulture, that has shown up in our list of Spider-Man characters. In addition we just introduced some new elements in our example that need some explanation for readers new to Clojure.</p>

    
    
    
        
        Uh oh, Vulture must be up to no good if he shows up here. © Marvel Studios</h4>
        
        
    </figcaption>
</figure>

The first is the use of square brackets [], which indicate a list. This is a collection of elements, in this case of spider-man-profile</code> and vulture-profile</code>, that unlike a set, can have multiples of the same element. </p>
The other new type of element we use is the form of using parentheses along side def as in (def spider-man-characters ...)</code>. Expressions of these type, called symbolic expressions, or s-expressions for short, are a characteristic of the Lisp family of languages to which Clojure belongs to. In a Lisp, parts of the program are either atoms, such as 5</code>, "Peter Benjamin Parker"</code>, true</code>, or an s-expression where the first element between parens is a function and the rest are parameters. For example (+ 1 3)</code>. While atoms evaluate to themselves, the s-expressions evaluate to a function with the given parameters. In the case of (+ 1 3)</code> they should evaluate to 4</code>. You can also nest s-expressions, such as (- (+ 1 3) 2)</code>, which will evaluate to 2</code>.</p>
You might be thinking, "Wait, if everything is either an atom or an s-expression, what kind of villainous things are those strange brackets that one has to use to create a set, list or map!". For all the simplicity in Clojure, it does make use of some syntactical sugar. Lists can be written [spider-man-profile vulture-profile]</code> as a shorthand for the s-expression (list spider-man-profile vulture-profile)</code>. Similar functions exist for maps and sets as well. </p>
Much like Spider-Man, who for all his powers still has to struggle with juggling a school and a job and has to make practical decisions, Clojure has to make them as well. In this case because certain things, such as maps, sets and list are used so often, it uses a shorter syntax for creating them. This does makes the language slightly more complex, but in the author's view, it pays off. </p>
Another matter of practicality of course is that while we can nest the two profiles directly into a list, we can create variables for them to associate. The def</code> function does exactly this, and it also ensures they become part of the current namespace. For example, if the current namespace is spider-man-spec.core</code> then a def</code> of vulture-profile</code> can be referred to as spider-man-spec.core/vulture-profile</code> from other namespaces, and simply vulture-profile</code> in the current namespace. This allows us to break up the overall data in smaller parts to use.</p>
Now we finally described the profiles of both Spider-Man and Vulture, but are they correct? The library of clojure.spec uses the notion of a spec for this. A spec is simply a function on a single parameter that returns a truthy value (in most cases a true</code> if the spec holds, false</code> if the spec does not hold).</p>
In essence this allows for many existing functions to be used as specs. For example the already existing function string?</code> checks whether a particular value is a string or not. </p>
In order to check whether a value is valid for a particular spec we can use the s/valid?</code> function. Here the s</code> stands the namespace of the spec library clojure.spec.alpha</code>, so by calling s/valid?</code> we are calling the valid?</code> function of this particular namespace.</p>
(s/valid? string? "</span>Spider-Man</span>")
</span></code></pre>
The above function call will checking if "Spider-Man"</code> is indeed a string, and return true</code> if it is. On the other hand if we check whether a number is valid for this spec, using (s/valid? string? 6)</code> we instead get false</code> returned. </p>
Another way to use a spec, is to explain why a value is wrong. For example, we can call the function explain-data</code> on with the spec and an incorrect value, to get a map back with an explanation. The function call: </p>
(s/explain-data string? </span>6</span>)
</span></code></pre>
would result in the map:</p>
{</span>:val 6 :predicate :clojure.spec.alpha/unknown</span>}
</span></code></pre>
Now the above example clearly shows the value on which the spec has failed, but it denotes the predicate as unknown with :clojure.spec.alpha/unknown</code>. The solution to this is to provide a name for the spec, which the system can use to pin point if things fail. We can register any spec using the function s/def</code>. For example the functions: </p>
(s/def </span>::name </span>string?)
</span>
</span>(s/def </span>::real-name </span>string?)
</span></code></pre>
will register the two specs under the keys :name</code> and :real-name</code> in the current namespace, i.e.: under spider-man-spec.core/name</code> and spider-man-spec.core/name</code> respectively.</p>
Now if we would aim to explain why the spec :real-name</code>  does not allow the value 6</code>, it would return the explanation:</p>
{</span>:val 6 :predicate :spider-man-spec.core/real-name</span>}
</span></code></pre>
where the predicate now identifies the spec that was not fulfilled.</p>
Specs can also be created in other ways. For example a set of values indicating the correct values can be used as a spec.</p>
(s/def </span>::identity </span>#{</span>::secret ::publicly-known</span>})
</span></code></pre>
The above code defines a spec for identity as having two possible values: either ::secret</code> or ::publicly-known</code>. </p>
Specs can also be defined for collections as well. The specs for current- and former affiliations: </p>
(s/def </span>::current-affilications </span>(s/coll-of string? </span>:kind </span>set?))
</span>
</span>(s/def </span>::former-affilications </span>(s/coll-of string? </span>:kind </span>set?))
</span></code></pre>
These specs describe that that both current- and former have to be sets of strings. The affiliations part of a profile is actually map containing both current- and former affiliations. This is defined as the spec:</p>
(</span>s/def </span>::affiliations </span>(</span>s/keys </span>:req </span>[</span>::current-affiliations</span>] </span>:opt </span>[</span>::former-affiliations</span>]) )
</span></code></pre>
which makes it requirement for affiliations to contain current-affiliations, but any former affiliations are optional.</p>
For checking whether Spider-Man has a valid profile we can use the s/valid?</code> function again. We use the following code to do just that:</p>
    (let [spider-man-affiliations (</span>:spider-man-spec.core/affiliations </span>spider-man-profile)]
</span>      (s/valid? </span>:spider-man-spec.core/affiliations </span>spider-man-affiliations))
</span></code></pre>
The let</code> form is new here, but what it essentially does is deconstructing the while spider-man-profile</code> and associating its affiliations temporarily the spider-man-affiliations</code>. This allows us to use a shorthand when calling functions, instead of writing out everything in a single line.</p>
While this value is also valid according to the spec, as the spec and the value we are checking gets more complex, it could also be useful to gather the exact value that has passed the spec. In such cases we can use s/conform</code> to gather these. The call: </p>
(s/conform </span>:spider-man-spec.core/affiliations </span>spider-man-affiliations)
</span></code></pre>
Returns the map:</p>
#:spider-man-spec.core{</span>:current-affiliations </span>#{"</span>Avengers</span>"}, </span>:former-affiliations </span>#{"</span>The Outlaws</span>" "</span>Secret Defenders</span>" "</span>New Fantastic Four</span>"}}
</span></code></pre>
Note that this is a namespaced map, which is a feature that allows us to refer to the keywords inside a map more efficiently, instead of writing them all out in each case.</p>
The final aspect of each profile, the power grid, is also something that can be given a spec. Each of the powers can only take a whole number value from 1 until 7. We can specify this with the follow spec:</p>
(s/def </span>::power-value </span>(s/and pos-int? #(>= % </span>1</span>) #(<= % </span>7</span>)))
</span>
</span>(s/def </span>::durability ::power-value </span>)
</span>(s/def </span>::energy ::power-value </span>)
</span>(s/def </span>::fighting ::power-value </span>)
</span>(s/def </span>::intelligence ::power-value </span>)
</span>(s/def </span>::speed ::power-value </span>)
</span>(s/def </span>::strength ::power-value </span>)
</span></code></pre>
Note that we use the function s/and</code> to combine three specs: that the value should be a positive integer, greater than or equal to 1 and less or equal to 7. Such a combined spec can then be (re-)used like any other.</p>
We can combine all the previous specs together to specify a profile: </p>
(s/def </span>::profile </span>(s/keys </span>:req </span>[</span>::name ::real-name ::identity ::affiliations ::power-grid</span>] ) )
</span></code></pre>
For this spec, both Spider-Man and Vulture are valid profiles. However, this is a problem, as it does not allow us to differentiate between a hero and a villain. Of course we do not want to get Vulture get into the same places as Spider-Man can. We must fight him, much like Spider-Man, but in our own way: by creating a spec for which the Spider-Man profile is a valid value, but not that of Vulture.</p>

    
    
    
        
        Spider-Man vs Vulture © Marvel Entertainment</h4>
        
        
    </figcaption>
</figure>

While we can make a separate requirement that only persons with the name "Spider-Man" can fulfill our new "hero-spec" this might be too restrictive. Instead we are going to spec an Avenger profile, so Spider-Man and all his friends can join in, while villains such as Vulture are kept out.</p>
The requirement for an Avenger in our system, is that any-one with the current affiliation of "Avengers"</code> is an avenger. We can describe this requirement as a spec, using a function defined for this:</p>
(defn is-avenger? [profile]
</span>  (contains? (</span>::current-affiliations </span>(</span>::affiliations </span>profile) ) "</span>Avengers</span>")
</span>  )
</span>
</span>(s/def </span>::avenger-profile </span>(s/and </span>::profile </span>is-avenger? ) )
</span></code></pre>
Now we can check whether a profile is a valid Avenger, which will be true for Spider-Man but not for Vulture. Finally, we can get rid of this villain that showed up in our tutorial. In addition, this spec will also make sure that all current members of the Avengers be valid, so Spider-Man can fight freely alongside of them. </p>

    
    
    
        
        Our spec answers the question posed in this cover: Spider-Man is indeed an Avenger © Marvel Entertainment</h4>
        
        
    </figcaption>
</figure>

So there we have it, a brief look at using the spec library to validate data. There are many things that I have not touched, such as the ability to generate values based on the Spec, other ways to compose a spec, etc.</p>
Nonetheless I hope this article gives a solid introduction, and maybe an interest to using the spec library, even if one does not have a Clojure or even a heavy programming background. The source code snippets are available at: Spider-Man-Spec</a>.</p>
If you have a data validation problem, by all means take a swing at it with the Spec library. I am convinced that the results you will get will be nothing short of spectacular.</p>


The Art of Choosing a Programming Language
2018-03-18T00:00:00+00:00
Programmers, like professionals in other fields, are passionate about their tools. One of the main elements in the toolbox of coders are programming languages. They allow their users to express solutions through code to tackle a large variety of problems in many domains. </p>
Programming is also an art, as described in the article by Donald Knuth titled Computer Programming as an Art</a> and in certain aspects of programming languages can be seen as art styles.</p>
As can be expected with many things that people are passionate about, whether viewed as a tool or an art style, coders can bond or argue about programming languages. Like philosophers of old, these discussions can go quite into depth, but to the outsider the arguments made or the sentiments behind them can be quite opaque. </p>

    
    
    
        
        If programming languages existed back then, I am sure they would be a hotly argued topic. School of Athens by Raphael</h4>
        
        
    </figcaption>
</figure>

Here I hope to shed some light to the casual observer on what makes programmers passionate about these languages and why some prefer one over the other. Such analysis can be quite subjective, and very much dependent on the writers experiences and preferences, but I will try my best to give an impartial overview.</p>
In theory many general purpose programming languages are capable of doing the same things. The most commonly used programming languages are Turing complete, meaning that they can all simulate the workings of any Turing machine. Without getting into the full description of what a Turing machine is, for the reader unfamiliar with the concept, this means that any of the languages can express programs for similar tasks.</p>
There are thousands of programming languages. Some older, and going back to the 50s, 60s and 70s and with considerable use still. Others have been released as recently as the last 10 years, and have gained considerable following. Given as I have mentioned that theoretically all these languages can do the same things, one could wonder why new languages are designed.</p>
History</h2>
Historically, the original computers were instructed by a pure machine language, for example 0s and 1s. Writing programs this way can be tedious and error prone, and the results code can be very difficult to read. This is one of reasons why assembly languages were created. These are languages that are still very much tied into the instruction set of a particular machine, but in a more human readable form, where symbolic names are given for machine instructions. These would be then translated to the pure machine language, to instruct the machine.</p>
While reading and writing programs becomes easier this way, using assembly languages still has disadvantages. First, these languages are still very much tied to the hardware. Different instruction architectures can mean that a program for the same goal would have to be written differently for each architecture. Second, for many the instructions that one has to write this way are still very low level. The argument is made that with a better set of abstractions over assembly, programs can be written in a better way. A program written with such abstractions could be translated, compiled, to the required machine code specific for the required architecture.</p>
The question of which abstractions need to be utilized is at the heart of why there are so many different programming languages. People have different ideas on what these abstractions might be, what the benefits and drawbacks of applying them are. This is at the heart of why people design and use newer programming languages. In the following sections we go through some of the aspects on these abstractions.</p>

    
    
    
        
        Much like with ancients wonders of next to a modern city, with programming languages old also gives rise to the new, and often co-exists with it. The Giza Pyramids © Robbster1983</h4>
        
        
    </figcaption>
</figure>

Paradigms and Style</h2>
As mentioned before, there are different opinions on how programs could be constructed. There are various subjects about on which people have opinions about: how the code is organized and how it is executed, among other elements. This is very much similar to how art styles function. For example the same subject can be painted in two differing styles.</p>

    
    
    
        
        The Last Supper (Leonardo da Vinci) one of the most famous Renaissance style paintings.</h4>
        
        
    </figcaption>
</figure>


    
    
    
        
        The Last Supper (Tintoretto) depicts the same subject but in a Mannerist, proto-Baroque style.</h4>
        
        
    </figcaption>
</figure>

Programming languages can be be classified on the different styles, programming paradigms</em> based on the common elements in the approaches.  Some paradigms include:</p>
Imperative</h3>
Imperative code can be seen as a set of commands for the computer to perform. This type of paradigm matches very strongly with how computer hardware is working, as nearly all computer hardware is designed the execute machine language, which is in itself is written in imperative style. </p>
Procedural</h3>
One of the ways one can structure a program is to group together a series of commands. These groups, procedures</em>, can then be called, used or reused as a single entity.</p>
Object-Oriented</h3>
Object oriented code uses the notion of objects</em> to organize code. An object is an encapsulation of related state and behavior. For example, consider a software that needs to represent a vehicle. The elements of the state that describe the object, such as colour</em> and make</em>, are called attributes.  Various functionality related to the object, such as calculating the price of the car, are called methods. These concepts allow reuse, as the objects for a car and a motorcycle can share functionality. </p>
Declarative</h3>
In declarative programming, one describes, or more aptly declares what the problem is as opposed to detailing the steps on how to solve it. This contrasts with imperative programming, where one gives the instructions on how to solve it directly. </p>
Functional</h3>
Functional programming is one form of declarative programming where programs are constructed using functions</em>, which are analogous and inspired by to mathematical functions. The intention is that these functions are ideally side effect free: their output is dependent solely on their input. This can make code easier to understand and allows for easier use of code written this way. </p>
Logic</h3>
The logic paradigm is based around expressing code as a set of logical axioms. These axioms can then be used as a from of knowledge base to derive new knowledge and query. The programs themselves then can be posed as a query in this system. For example, if the knowledge is defined with the axioms "Tweety is a bird" and "Birds are animals", the system should be able to answer the queries: "Is Tweety and animal?". </p>
A language can focus on supporting a particular paradigm heavily or have a strong preference for it. For example Haskell</a> or Clojure</a> lean quite heavily on the functional paradigm, while Prolog</a> is one of the main logic programming languages. Others, provide an explicit merge of various methodologies, such as Scala</a> that combines elements of object orientation and functional programming.</p>
Preference for a particular language can go beyond the programming paradigms used. Syntax, the structure of how code is written, can matter quite a bit for person's view on a particular language. For example Python</a> uses indentation for managing the control flow of the code, as opposed to symbols in other languages.</p>
Such preference can go even beyond the actual code itself to the tools one uses to write. While any text editor for editing text can often suffice, people can have differing expectations with regards to integrated development environments (IDEs) or other tools to edit and analyze the code. The lack or existence of specific tooling can also be a factor when deciding between languages.</p>
Available Code and Libraries</h2>
Most coding is done with a particular purpose in mind, and it is rarely the case that the programmer can build everything from the ground up for such a task. In order to build interesting programs, one has to utilize existing knowledge, much like someone would utilize knowledge in a library to come to new insights.</p>

    
    
    
        
        The Bibliotheca Alexandrina. Photo © Carsten Whimster licensed under CC BY 3.0 https://creativecommons.org/licenses/by/3.0/</h4>
        
        
    </figcaption>
</figure>

Existing code can be used as a foundation from which the program can be built. Roughly speaking existing code comes in three main forms. It is either being part of the language (often called the standard library of the language), some external libraries extending the language for a particular purpose, or an existing code base of the application that one can improve upon.</p>
The standard library contains various functionality included with the language itself. For example ways of manipulating files, various connection protocols, support for certain file formats, etc. Of course it is very much helpful if particular support for a certain feature that aims to use is already available with the language itself. This means less code to write and connect. On the other hand there is also some tension with regards to including too many features in the standard library, especially if certain parts of it become outdated, which enlarges the language and makes it more unwieldy. </p>
The external libraries that one can use in a language can also influence the choice of a language. Certain languages have a lot of library support for specific tasks. For example Python</a> has a large and active following in the Data Science community. Other languages have a lot of support for many different tasks simply due their age and user base such as Java</a>. By using libraries one does not need to implement certain features from scratch but can reuse existing work and focus on their specific problem at hand.</p>
Finally, not all development starts from scratch, often one has to make additions or improvement to an existing program, in which case the choice of the language has already been made. While a rewrite of the code can often be tempting, linking between two code programming languages is not always trivial. It is often a good idea to continue with an existing language.</p>
There are some exceptions to this as some languages have been designed with the ground up to inter-operate with other languages. A good example of this is Clojure</a> has great interop with Java</a> and/or JavaScript</a>. This allows it to leverage existing libraries already written, and makes it much more attractive to use. </p>
Existing Knowledge</h2>
Writing code is rarely trivial, and neither is learning new programming languages. Although previous experience helps, especially when dealing with languages with known paradigms, due to slight or large differences it can take a while to get used to the new language and libraries. With constantly looming deadlines and pressure to deliver, it can make sense to minimize the work that needs to be done. It is perfectly valid to work with a language that one already knows.</p>
Curiosity</h2>
On the other hand learning a new language, especially in a new paradigm or other innovative features, can be quite interesting. It not only allows for work on existing code written in the new language but it also gives insights in how to program which is beneficial as a programmer in general no matter what language he is using.</p>
Speed</h2>
As mentioned earlier, commonly used programming languages are abstractions over machine code that can do more of less the same thing computationally. What abstractions are used however can influence the speed of executing the program, as well as the time of translating the code in the programming language to machine code. </p>
A common abstraction that can influence the speed of executing the program is how memory is managed. During the running of a program certain information needs to be stored. A way to do this is to allocate space in the computers memory, keep it around while needed and remove it afterwards. This latter portion, can be quite difficult to manage manually, as if one does it prematurely the program might crash or have other bugs. Not removing it would fill the memory with garbage, which makes the program use up more and more memory till it crashes.</p>
A solution to these problems is automatic garbage collection: a way for the computer to automatically manage and clean up memory. While this is a good solution in many cases, this process comes with an overhead, and can be unpredictable when the time and resource consuming cleanup happens. In most cases this overhead is trivial to pay for eliminating a whole suite of potential bugs. However in certain scenarios, such as real-time high performance games, it could be too much to pay.</p>
The other issue of speed, translating the code from the programming language to machine code, can also be a consideration. Development requires making changes to code and checking whether the changes work. If the process of getting feedback takes a long time, due to these translations, it can destroy a programmers productivity. Golang</a> is a language that is explicitly designed for fast compilation.</p>
Safety</h2>
Safety is in many cases the flip side to the speed argument. Certain abstractions cost you in speed but provide you with safety in return. Different languages tend to make different trade-offs with this regard. For example one of the relatively new languages, Rust</a> aims at focus on zero cost abstractions: abstractions with little to no run-time performance penalty.</p>
One contentious aspect of safety is the use of type systems. Types allow the coder to specify various categories, such as numbers, persons, cars, etc as well as their requirements to be fulfilled within the context of the program. Types can be checked both statically, before the system is run, or dynamically, during the running of the program. Some people swear by very expressive type systems: where types can specify very detailed features of the things the program wants to represent. This then can be used for checking code for correctness, both before and during the running of a program, as well as documentation. On the other hand type checking is not free: it can make translating the compilation into machine code a much slower process. Some people also consider the writing and checking of types themselves very cumbersome during initial development, where quick iteration can be slowed down by specifying detailed types.</p>
There is a whole spectrum of possible stances with regards to type systems. For example, certain languages such as Haskell</a> and Idris</a> are designed from the ground up with very expressive type systems that are statically checked. Others, for example Dart</a> which started off as having optional types but adds mandatory types in the latest iteration to help with tooling, take a more balanced approach. Golang</a> explicitly has a static, but minimalist, type system that allows for fast compilation.  There are also languages, such as Clojure</a> that instead of static types, use contract systems to ensure safety at run-time and allow for documentation and testing.</p>
Deployment</h2>
While most general purpose programming languages can be made to run in all environments, they are not always available. In certain environments, such as mobile or on the web, only specific languages are supported. For example on Android Java</a> and Kotlin</a> are officially supported, while on the web JavaScript</a> is the current Lingua Franca of the web. This means that it can be quite a herculean effort to make other languages work in such environments, and going with the most supported option is easier. </p>
The way certain languages can get around on this hindrance is by using the more commonly supported language as the target to translate into. For example ClojureScript</a> compiles into JavaScript</a>. And in some cases, other developers have made the effort to get frameworks up and running that allow the use of a different language, such as the use of React Native</a> and Flutter</a> that allow the use of JavaScript</a> and Dart</a> respectively to develop mobile applications. </p>
The Team and Beyond</h2>
One final aspect of choosing a programming language, which can be surprisingly significant, is which language is beneficial to the team, as opposed to an individual developer. Different teams bring different expertise to the table, and while most professionals are often quite willing and able to use a new language if it is most suited to the task at hand, this can still be a cost that might be better spent on developing the application. From an employers perspective it can also often be beneficial to stick to more commonly used languages as it can be easier to find future employees versed in the language used. On the other hand, there are many professionals that would be quite willing to jump on the chance of using the latest programming languages, in which case the choice for a newer or more niche language can be a competitive advantage from a recruiting perspective.</p>
Conclusion</h1>
I hope this article gave some insight on why programmers pick and argue about programming languages. Despite all the various differences and arguments it is also very important to note, that great software has been written in many different languages, that is both excellent code and solves important problems. And while picking the right tool for the job is an important, it can be just an aspect of the art of solving problems with code.</p>


Data Interlinked
2018-02-18T00:00:00+00:00
This article contains some very minor spoilers for the Blade Runner 2049 movie. If you have not seen it yet, by all means do, it is really good.</em></p>
"And blood-black nothingness began to spin... A system of cells interlinked within cells interlinked within cells interlinked within one stem... And dreadfully distinct against the dark, a tall white fountain played."</p>

    
    
    
        
        Pale Fire by Vladimir Nabokov © Berkley Medallion</h4>
        
        
    </figcaption>
</figure>

The above is a quote from the poem Pale Fire</em>. It occurs in the novel titled Pale Fire</em> by Vladimir Nabokov which has been recently used in the movie Blade Runner 2049</em>. In the movie it was part of the Baseline test, a way to test the emotional response of a Replicant. The reason they undergo this process is because their creators fear that the connections they might make in their lives would give them emotions that would interfere with their intended purpose.</p>

    
    
    
        
         Blade Runner 2049 Poster for the Movie © Columbia Pictures</h4>
        
        
    </figcaption>
</figure>

In life, such relationships always surround us. They are not just between people, but in our work, in our beliefs, in our art and in the knowledge we represent.</p>
Suppose we intend to describe the link between the movie Blade Runner</em> and the book Pale Fire</em>. We can summarize this information with a number of facts.</p>
Blade Runner is a movie. 
</span>Blade Runner has a character named K.
</span>K is a Replicant.
</span>Replicants must pass a Baseline Test.
</span>Baseline Test is based on the poem Pale Fire.
</span>Pale Fire is written by Vladimir Nabokov.
</span></code></pre>
The above facts show that the links that can tie together various pieces of knowledge. One can trace the connections from a simple description of a movie, released in 2017, to the author Vladimir Nabokov, as was intended by the writers of the movie.</p>
Although the above recitation of facts is easy to follow, from a knowledge representation perspective one can find some issues with it. </p>
First the description is imprecise. As the Blade Runner</em> could refer to the newer Blade Runner 2049</em> movie as opposed to the 1982 original titled Blade Runner</em>. </p>

    
    
    
        
        Movie poster for the first Blade Runner Movie © 1982 The Ladd Company</h4>
        
        
    </figcaption>
</figure>

Second the set of facts is incomplete. The poem Pale Fire</em> is indeed written by Vladimir Nabokov, but it is presented in the book Pale Fire</em>, also written by Nabokov as the work of the fictional poet John Shade</em>. The set of facts here fails to make the explicit distinction between Pale Fire (poem)</em> and Pale Fire (book)</em>, and that the poem is contained in the book. </p>
Third, and perhaps most importantly, the above list of facts relies a lot on the users grasp of the English natural language. For a program, it can be surprisingly difficult to understand the relationships such as "is a", "is based on", "named", etc between the various elements in the text.</p>
These issues seem somewhat nit-picky, as this information can be derived from the rest of the article. However this means that the knowledge in the summary does not stand on its own. If those facts are detailed without the rest of the article, or if the reader of them is a machine, and not a person that can easily add some context, they might lead them to incorrect or insufficient conclusions. They might get wrong information that the 1982 movie Blade Runner has a character named K, or fail to see the link that poem is contained in the book by the same author. And although in the case of Blade Runner, these issues might seem small, this is different if the knowledge relates to financial, legal or clinical domains. Here, mistakes or omissions can be costly.</p>
Having a larger list of more detailed facts can help with these issues, but to a certain extent they still remain due to the ambiguity of the natural language. In addition the fact that is often very easy to skip over implicit details. This is especially true for the issue of a computer not being able to make (enough) sense of this information. </p>
A proposed solution to these issues is Linked Data</a> and in particular Resource Description Framework (RDF)</a>, with which Linked Data data can be expressed. These technologies allow us to represent the above facts in a more formal and precise way, that can make it both human and machine read- and write-able. </p>

    
    
    
        
        Resource Description Framework (RDF) Logo © W3C</h4>
        
        
    </figcaption>
</figure>

One significant feature of RDF is that requires precise naming. Many elements of it are either a International Resource Identifier (IRI)</a> or some raw data-types. Good examples of the former are URLs, such as the link to this website: http://www.newresalhaider.com , that allows one to find a web resource. Examples of the later are texts or numbers, such as "Blade Runner" or 15 respectively.</p>
The other significant feature of RDF is that most knowledge is represented as a set of facts, where each fact is expressed as subject, predicate object triples. For example the fact "Blade Runner is a movie" is expressed with the subject "Blade Runner" the predicate "is a" and the object "movie".</p>
Putting this together in RDF (using the Turtle</a> notation) you would get a triple such as:</p>
<http://www.newresalhaider.com/ontologies/bladerunner/blade-runner> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://www.newresalhaider.com/ontologies/bladerunner/movie></span>.
</span></code></pre>
This example is an RDF way of saying "Blade Runner is a type of movie" or alternatively "Blade runner is a movie". This type of representation shows us a couple of benefits. First we are now being more precise as each element in the triple can refer to one specific resource, for example Blade Runner or Movie, where there IRI makes sure we do not necessarily confuse the term with anything else. Second this also shows off the fact that you can link to resources from different places: the predicate "type" is from a completely different domain. This allows us to re-use knowledge that has already been defined. As one can expect saying something is of a "type", for example an apple is a type of a fruit, is actually very common. This is one of the main strengths of what makes Linked Data so powerful, one can re-use knowledge already stated. </p>
Typing out the full IRI each time can be pretty bothersome, and it does not help the readability either. Thankfully we can define a common prefix we use separately, and just write the last part of the IRI in each case. In this case we define a base prefix and we refer to subject and object by "<#blade-runner>" and "<#movie>" respectively.</p>
@base </span><http://www.newresalhaider.com/ontologies/bladerunner></span> .
</span><#blade-runner> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <#movie></span>.
</span></code></pre>
We can do similar things when linking elements that have been already defined elsewhere. In this case we define a prefix to use as an abbreviation while writing:</p>
@base </span><http://www.newresalhaider.com/ontologies/bladerunner></span> .
</span>@prefix </span>rdf: </span><http://www.w3.org/1999/02/22-rdf-syntax-ns#></span> .
</span>
</span><#blade-runner> </span>rdf:</span>type </span><#movie></span>.
</span></code></pre>
In practice, "rdf:type" as a predicate is so common that there is an even simpler notation. We can use 'a' as a predicate, which is in line with what we intend to express: "Blade Runner is a movie".</p>
The resulting RDF facts look as follows (note that the rdf prefix could be omitted here as the "a" abbreviation does not make it necessary):</p>
@base </span><http://www.newresalhaider.com/ontologies/bladerunner></span> .
</span>@prefix </span>rdf: </span><http://www.w3.org/1999/02/22-rdf-syntax-ns#></span> .
</span>
</span><#blade-runner> </span>a </span><#movie></span>.
</span></code></pre>
If we aim to write something that is just a text as a subject, say when referring to the title of a movie , we can do that as well:</p>
@base </span><http://www.newresalhaider.com/ontologies/bladerunner></span> .
</span>@prefix </span>rdf: </span><http://www.w3.org/1999/02/22-rdf-syntax-ns#></span> .
</span>
</span><#blade-runner> </span>a </span><#movie></span>.
</span><#blade-runner> <#title> </span>"</span>Blade Runner 2049</span>".
</span></code></pre>
With this way of writing, we can actually rewrite our original set of facts as follows:</p>
@base </span><http://www.newresalhaider.com/ontologies/bladerunner></span> .
</span>@prefix </span>rdf: </span><http://www.w3.org/1999/02/22-rdf-syntax-ns#></span> .
</span>@prefix </span>foaf: </span><http://xmlns.com/foaf/0.1/></span> .
</span>
</span><#blade-runner> </span>a </span><#movie></span>.
</span><#blade-runner> <#title> </span>"</span>Blade Runner 2049</span>".
</span><#blade-runner> <#has-character> <#K></span>.
</span><#K> </span>foaf:</span>name </span>"</span>K</span>".
</span><#K> </span>a </span><#replicant></span>.
</span><#replicant> <#must-pass> <#baseline-test></span>.
</span><#baseline-test> <#based-on> <#pale-fire-poem></span>.
</span><#pale-fire-poem> <#included-in> <#pale-fire-book></span>.
</span><#pale-fire-book> <#written-by> <#nabokov></span>.
</span><#nabokov> </span>foaf:</span>name </span>"</span>Vladimir Nabokov</span>".
</span></code></pre>
With this version we suddenly defined our list of facts in a more formal manner than previously. This makes it much more simpler for machines to understand this set of facts. In fact we actually used the Friend of a Friend (FOAF) ontology</a> to use the notion of name that is also used when talking about relationships between people. In fact, one could argue that using an existing movie dataset, such as the Linked Movie Database</a> would have been even better, which we will leave as an exercise for the reader.</p>
Hopefully I could show a glimpse of the possibilities the Semantic Web</a> for which Linked Data forms the basis, with this example. Of course the above is just scratching the surface of what it can be done with RDF, Linked Data. With each addition, our set of facts could grow. One could go beyond a single movie and build a document of poems that are references in movies, or a knowledge base of the Blade Runner franchise. It might be easier than one expects, due to the fact that knowledge, much like people are...</p>
Interlinked.</p>


The Trouble with Triples
2018-01-28T00:00:00+00:00
In the Star Trek episode "The Trouble with Tribbles" the crew of the starship Enterprise encounters creatures called Tribbles. They are cute, simple creatures of mysterious origin that seem harmless at first but when they multiply the pose a big problem for the ship and the crew.</p>

    
    
    
    
    
        
        Tribbles</h4>
        
        
        Tribbles © 1967 Paramount Pictures
        
    </figcaption>
</figure>
Representing and reasoning with knowledge have surprisingly similar problems. A single fact on its own is a relatively straightforward affair. A fact, such as "Tribbles are cute"</strong> can be represented with only three parts of a triple: a subject Tribbles</strong>, a predicate are</strong> and an object cute</strong>. Things can get quite a bit more difficult when there are more facts/triples: "Tribbles are round"</strong>, "Tribbles are furry"</strong>, "Tribbles originate from Iota Geminorum IV"</strong>, and other millions of facts that one could have about such a species. This is especially true when one takes into the account that the fact that knowledge can be interlinked "Iota Geminorum IV is a planet"</strong>,  "Iota Geminorum IV is also known as Fafniri"</strong>, "Iota Geminorum IV is also known as Tribble Prime"</strong>.</p>
This makes representing and reasoning with facts a non-trivial process. A system that holds all this knowledge should be able to answer a query such as "Do Tribbles originate from Fafniri?"</strong> with a yes, based on the facts "Tribbles originate from Iota Geminorum IV"</strong>, "Iota Geminorum IV is a planet"</strong> and "Iota Geminorum IV is also known as Fafniri"</strong>, even in the context of millions of other triples. </p>
Another interesting issue with representing facts is the context of the information. To us the viewers, and initially to the crew of the Enterprise, Tribbles look like harmless and adorable creatures. To the Klingons they are an ecological menace and their mortal enemies. How such "knowledge about knowledge" is represented and used is often a challenging problem.</p>
Various technologies have been proposed to deal with the above-mentioned issues. The Semantic Web technologies</a> of Linked Data</a> and Ontologies</a> in particular have been designed around solving many of these problems. Nonetheless there is room for improvement. In the future I hope to be able to explain how these techniques can be utilized and perhaps lessen the pain points that currently surround (the use of) them.</p>


Introduction
2018-01-14T00:00:00+00:00
Welcome to the first post on my blog on which hopefully many will follow. My intention is to write about knowledge representation, reasoning, AI and coding, in an easily digestible but in depth-way.</p>


Legal Regulatory Compliance Within The Financial Domain.
2017-12-31T00:00:00+00:00
Regulatory compliance is the goal of an organization to ensure it complies with all the relevant policies, regulations and laws. Failure to comply can have grave consequences for an organization, with huge fines and penalties being imposed. Regulatory pressure has been increasing over the years, with new and more complex regulation being enacted. This is especially true for the financial sector, where in the wake of the financial crisis, new policies and laws where enacted, both internal and external, to prevent a similar crisis happening in the future.</p>
These new and changing policies and laws give a huge challenge for financial organizations. These issues get compounded by fact that in a global world, internal, national and international regulations create an interwoven set of rules that need to be interpreted and applied on various financial products and services. Ensuring this compliance, is a time and expertise intensive task that often needs to be done manually. Automating these tasks is a huge challenge due to the fact that more traditional techniques are often ill equipped to tackle such an ever changing and complex domain in a way that is transparent to the domain experts and regulators.</p>
In this project, the goal is to find innovative techniques and solutions to the issue of regulatory compliance in the legal and financial domain.</p>


Smart Industry Software
2017-12-30T00:00:00+00:00
innius</a> is a powerful smart industry software platform for industrial machine manufacturers and industrial machine users. </p>


Ontology Enabled Clinical Software
2017-12-29T00:00:00+00:00
The time of clinical health professionals is incredibly valuable. Software can alleviate the burden of physicians, nursers and other health experts and allow them to provide better, more effective health care to patients. </p>
Creating, maintaining and using software in clinical environments is a difficult process. Not only it is paramount that such software needs to be correct, it has to have an understanding of highly domain specific knowledge about clinical processes in order to provide beneficial support to health professionals.</p>
Semantic Web technologies, in particular anthologies enable better clinical software. Ontologies allow for a formal and explicit way to model knowledge that is understandable for both clinicians and machines alike. </p>


Semantic Aware Software Analysis
2017-12-22T00:00:00+00:00
Software bugs and errors have a monumental negative impact on society. Not only can they have huge monetary cost, ranging into billions of dollars annually in the United States alone, but in some cases can even lead to disasters that lead to the loss of life. Preventing such issues from happening requires a large amount of effort in designing and maintaining software. </p>
However designing and maintaining error free software remains a difficult problem. Not only is software getting more complex, its correct functioning often depends on highly specific domain knowledge of its purpose and its environment. Therefor manually checking the correctness of the software is expensive in both time and available expertise.</p>
A potential solution that was proposed for this is using machine understandable domain knowledge to help automate software maintenance tasks. Formal ontologies have been proposed and used in various fields to make domain knowledge explicit and usable for both humans and software. In particular during my thesis work, titled "An Ontology Based Framework for Specification Mining and Dynamic Program Analysis", the use of ontologies to help understand the working of a running program were successfully explored. In particular it was shown that for a category of bugs, general ontologies such as WordNet, can be used find issues issues from program traces.</p>


Integrating Existing Large Scale Medical Laboratory Data into the Semantic Web Framework
2014-10-27T00:00:00+00:00
Abstract</h1>
"Semantic Web technologies have shown to have great potential in many different domains, to facilitate knowledge representation, exchange and reasoning, in a formal and yet both human and machine understandable way. In particular, within the health domain, they enable knowledge integration and understanding by explicitly defining and linking concepts and relationships using ontologies to information within clinical knowledge bases. This additional metadata also allows for automated decision support and semantic based analytics to be implemented, that facilitate improved healthcare at a lower cost. Unfortunately many existing datasets in healthcare environments are still stored in relational databases, as opposed to using semantic technologies. Due to this, the link with explicit metadata is often lacking or non-existent. Furthermore, both the databases and the clinical terminologies can be considerably large, making the mapping and subsequent uses of the information a difficult process. In a full fledged decision support system the level and accuracy of the mapping can greatly influence the effectiveness of any subsequent analysis and decision support tasks. This is especially true in clinical scenarios, where very large and complex sets of terms need to be mapped to relational databases. In this paper we aim to provide a general approach for interlinking relational data with clinical ontology based metadata that allows for a fine grade evaluation, with respect to the mapping's impact on analytics. We evaluate our approach by mapping information from clinical terminologies, such as SNOMED CT, to a large laboratory dataset contained in a relational database, with the goal of creating a full fledged, semantically enabled, analytics and decision support system."</p>
Authors</h1>
Newres Al Haider, Samina Abidi,  William Van Woensel, Syed SR Abid</p>
Appeared</h1>
In 2014 IEEE International Conference on Big Data (Big Data)
Link</a></p>
Related Project</h1>
Clinical Semantics</a></p>


A Cross-Platform Benchmark Framework for Mobile Semantic Web Reasoning Engines
2014-10-19T00:00:00+00:00
Abstract</h1>
"Semantic Web technologies are used in a variety of domains for their ability to facilitate data integration, as well as enabling expressive, standards-based reasoning. Deploying Semantic Web reasoning processes directly on mobile devices has a number of advantages, including robustness to connectivity loss, more timely results, and reduced infrastructure requirements. At the same time, a number of challenges arise as well, related to mobile platform heterogeneity and limited computing resources. To tackle these challenges, it should be possible to benchmark mobile reasoning performance across different mobile platforms, with rule- and datasets of varying scale and complexity and existing reasoning process flows. To deal with the current heterogeneity of rule formats, a uniform rule- and data-interface on top of mobile reasoning engines should be provided as well. In this paper, we present a cross-platform benchmark framework that supplies 1) a generic, standards-based Semantic Web layer on top of existing mobile reasoning engines; and 2) a benchmark engine to investigate and compare mobile reasoning performance."</p>
Authors</h1>
William Van Woensel, Newres Al Haider, Ahmad Marwan Ahmad, Syed SR Abidi</p>
Appeared</h1>
In WODA '10 Proceedings of the Eighth International Workshop on Dynamic Analysis Link</a></p>
Related Project</h1>
Clinical Semantics</a></p>


A Comparison of Mobile Rule Engines for Reasoning on Semantic Web Based Health Data
2014-08-11T00:00:00+00:00
Abstract</h1>
"Semantic Web technology is used extensively in the health domain, due to its ability to specify expressive, domain-specific data, as well as its capacity to facilitate data integration between heterogeneous, health-related sources. In the health domain, mobile devices are an essential part of patient self-management approaches, where local clinical decision support is applied to ensure that patients receive timely clinical findings. Currently, increases in mobile device capabilities have enabled the deployment of Semantic Web technologies on mobile platforms, enabling the consumption of rich, semantically described health data. To make this semantic health data available to local decision support as well, Semantic Web reasoning should be deployed on mobile platforms. However, there is currently a lack of software solutions and performance analysis of mobile, Semantic Web reasoning engines. This paper presents and compares the mobile benchmarks of 4 reasoning engines, applied on a dataset and rule set for patients with A trial Fibrillation (AF). In particular, these benchmarks investigate the scalability of the mobile reasoning processes, and study reasoning performance for different process flows in decision support. For the purpose of these benchmarks, we extended a number of existing rule engines and RDF stores with Semantic Web reasoning capabilities."</p>
Authors</h1>
William Van Woensel, Newres Al Haider, Patrice C Roy, Ahmad Marwan Ahmad, Syed SR Abidi</p>
Appeared</h1>
publication = "In Proceedings of the 2014 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT)-Volume 01"
Link</a></p>
Related Project</h1>
Clinical Semantics</a></p>


Towards Guideline Compliant Clinical Decision Support System Integration in Smart and Mobile Environments: Formalizing and Using Clinical Guidelines For Diagnosing Sleep Apnea
2014-06-18T00:00:00+00:00
Abstract</h1>
"Assistive technologies in smart environments were developed in order to maintain and improve the quality of life of people with dementia or other health problems. In order to provide adequate support at the opportune moment, it is necessary to deploy ambient services, such as activity recognition and assistance planning. Clinical Decision Support Systems (CDSS) that implement clinical guidelines, allow for the right clinical decisions, such as diagnoses and treatment choices, to be made automatically based on patient data and other health information. While data derived from smart home services can be used in these CDSS, smart homes can be used to provide services related to clinical decisions. Mobile devices can be used in conjunction with the smart home services and a remote CDSS for sending notifications or retrieve data from wearable or built-in sensors. However, in a context where smart homes interacts with a remote CDSS, we must take into account mobility (e.g. outdoors, work), failure tolerance (e.g. connection issues with remote CDSS) and privacy concerns in order to provide minimum quality of service. Thus, CDSS must be locally deployed as a smart home service and on a mobile device. In this paper we investigate a scenario where due to the above mentioned reasons a clinical guideline compliant CDSS needs to be deployed on a mobile device and as a smart home service, where clinical data can come from either the patient (manual input) or the smart home and mobile sensors. In particular we focus on implementing and evaluating a guideline for the diagnosis of sleep apnea. Sleep apnea diagnosis is a well-suited task for this purpose as attributes for the execution of the guideline could be collected both from the patient and sensor data inside or outside the smart home environment. In order to illustrate the feasibility of CDSS as smart home service and on mobile device, the Sleep Apnea CDSS is validated on an Android smartphone and show promising results."</p>
Authors</h1>
Patrice C Roy, Newres Al Haider, William Van Woensel, Ahmad Marwan Ahmad, Syed SR Abidi</p>
Appeared</h1>
In The Twenty-Eighth AAAI Conference on Artificial Intelligence (AAAI-14), Québec City, Canada
Link</a></p>
Related Project</h1>
Clinical Semantics</a></p>


Execution Trace Exploration and Analysis using Ontologies
2011-09-27T00:00:00+00:00
Abstract</h1>
"Dynamic analysis is the analysis of the properties of a running program. In order to perform dynamic analysis, information about the running program is often collected through execution traces. Exploring and analyzing these traces can be an issue due to their size and that knowledge of a human expert is often needed to derive the required conclusions. In this paper we provide a framework in which the semantics of execution traces, as well as that of dynamic analyses, are formally represented through ontologies. In this framework the exploration and analysis of the traces is enabled through semantic queries, and enhanced further through automated reasoning on the ontologies. We will also provide ontologies to represent traces and some basic dynamic analysis techniques, along with semantic queries that enable these techniques. Finally we will illustrate our approach through an example."</p>
Authors</h1>
Newres Al Haider, Benoit Gaudin, John Murphy</p>
Appeared</h1>
In RV '11 International Conference on Runtime Verification
Link</a></p>
Related Project</h1>
Semantic Software Analysis</a></p>


An Approach for Modeling Dynamic Analysis using Ontologies
2010-07-12T00:00:00+00:00
Abstract</h1>
"In this paper we present the possibility of using an ontology based framework in order to model Dynamic Analysis techniques. This work relies on similar ideas applied to the case of Static Analysis, in which ontologies are used to represent some knowledge about the programs to be analyzed. In the approach proposed in this paper we describe how ontologies can be applied to Dynamic Analysis by modeling both the information collected from the system, as well as some requirements about the type of analysis to be performed. Both of these ontologies can be designed by integrating ontologies previously defined during the software development cycle, allowing for re-usability. Finally, these ontologies make it possible to reason about concepts related to Dynamic Analysis and offer tools that facilitate automation. This paper presents the main ideas of the proposed approach and illustrates them with an example related to Frequency Spectrum Analysis."</p>
Authors</h1>
Newres Al Haider, Benoit Gaudin, Paddy Nixon</p>
Appeared</h1>
"In WODA '10 Proceedings of the Eighth International Workshop on Dynamic Analysis"</p>
Related Project</h1>
Semantic Software Analysis</a></p>

Growing Yggdrasil, the World Tree, with Ash

Falsehoods AI Believes About Names

Will AI Get Rid of this Turbulent Priest?

They LiveView

Gathering Rust

The Great AI'Tuin

Into the Matrix

Livebook of Oa

A Mermaid.js timeline of the first appearance of Green Lanterns in the comic books that is created with the above code.</h4> </figcaption> </figure> Code blocks can do more than just set up our notebook. For example we can define a struct to describe data relating to a Lantern. </p>

Fellowship of Prolog

Objective</h4> You need to determine the walking order of seven characters: Aragorn, Frodo, Sam, Merry, Pippin, Gimli, and Legolas.</p>

AI at Home

Accidental Renaissance

A photo of a kitchen in renaissance style generated on mage.space.</h4> </figcaption> </figure> A generated photo of a car in a renaissance style got the right category predicted with a probability of 0.8643.</p>

A Linked Data Based Advanced Credit Rationale

Authors</h1> Newres Al Haider, Keng Ng, Ali Hashmi, Lauma Veidemane, Diederik Schut </p>

Dunes

Star Wars Trek

Raku Parrot

Data Oriented Domain Design

The Pharo Prince of Bel-Air

We did not define what we mean by FPWillSmith which brings us some options on what to do now.</h4> </figcaption> </figure>

Meditations in your Pocket

Navigating the text of Meditations in the app running on an Android emulator.</h4> </figcaption> </figure>

To Curry in League of Legends

CybOrg Mode

Folding and unfolding headlines in Org document using Spacemacs</h4> </figcaption> </figure>

Darmok in core.logic

The Probable Mystery Machine

Solving Financial Regulatory Compliance Using Software Contracts

Authors</h1> Newres Al Haider, Dilhan Thilakarathne, Joost Bosman</p>

Aesopica, Part 5: Blank Nodes

Exploring the Deep, Part 1: Introduction

Elements of Machine Learning</h2>

Data</h3> Data is the example information from which the machine learning aims to learn. In the previous scenario, the data is the labelled photos of pets.</p>

Model</h3> The model designates the whole process of using the data and transforming it into the goal of machine learning. In our example the model would be the full process that can take a given photo and transform it into a prediction of the pet depicted.</p>

Optimization Algorithm</h3> The optimization algorithm that describes the process of moving from one model to a better one. This allows for the learning to happen by moving from one model to a one that better solves the problem.</p>

Kinds of Machine Learning</h2> Machine learning is a large field with a wide variety of techniques and potential problems to solve. Generally techniques are differentiated by the data they input and output, as well as the specific type of problem they target.</p>

Prolog and the Price of Peace

Aesopica, Part 4: Basic Conversion to Other Formats

Aesopica, Part 3: Named Graphs

Aesopica, Part 2: Literal Values

Aesopica, Part 1: General Introduction

1884 fountain design depicting the story of the Fox and the Stork by Catalan sculptor Eduard Batiste Alentorn in Barcelona</h4> </figcaption> </figure>

Excavating a Common Treasure

Digging in the certain urban areas can easily lead to excavations done before any subsequent work is done. Romano-Celtic temple excavated in London.</h4> </figcaption> </figure>

Spec-stacular Spider-Man

The Art of Choosing a Programming Language

Much like with ancients wonders of next to a modern city, with programming languages old also gives rise to the new, and often co-exists with it. The Giza Pyramids © Robbster1983</h4> </figcaption> </figure>

The Last Supper (Tintoretto) depicts the same subject but in a Mannerist, proto-Baroque style.</h4> </figcaption> </figure> Programming languages can be be classified on the different styles, programming paradigms</em> based on the common elements in the approaches. Some paradigms include:</p>

Procedural</h3> One of the ways one can structure a program is to group together a series of commands. These groups, procedures</em>, can then be called, used or reused as a single entity.</p>

Objective</h4>
You need to determine the walking order of seven characters: Aragorn, Frodo, Sam, Merry, Pippin, Gimli, and Legolas.</p>

Authors</h1>
Newres Al Haider, Keng Ng, Ali Hashmi, Lauma Veidemane, Diederik Schut </p>

Authors</h1>
Newres Al Haider, Dilhan Thilakarathne, Joost Bosman</p>

Data</h3>
Data is the example information from which the machine learning aims to learn. In the previous scenario, the data is the labelled photos of pets.</p>

Model</h3>
The model designates the whole process of using the data and transforming it into the goal of machine learning. In our example the model would be the full process that can take a given photo and transform it into a prediction of the pet depicted.</p>

Optimization Algorithm</h3>
The optimization algorithm that describes the process of moving from one model to a better one. This allows for the learning to happen by moving from one model to a one that better solves the problem.</p>

Kinds of Machine Learning</h2>
Machine learning is a large field with a wide variety of techniques and potential problems to solve. Generally techniques are differentiated by the data they input and output, as well as the specific type of problem they target.</p>

The Last Supper (Tintoretto) depicts the same subject but in a Mannerist, proto-Baroque style.</h4> </figcaption> </figure>
Programming languages can be be classified on the different styles, programming paradigms</em> based on the common elements in the approaches. Some paradigms include:</p>

Procedural</h3>
One of the ways one can structure a program is to group together a series of commands. These groups, procedures</em>, can then be called, used or reused as a single entity.</p>