MoonScript is a dynamic scripting language that compiles intoLua. It gives you the power of one of the fastest scripting languages combined with a rich set of features.

classThingname:"unknown"classPersonextendsThingsay_name:=>print"Hello, I am #{@name}!"withPerson!.name="MoonScript"\say_name!

MoonScript can either be compiled into Lua and run at a later time, or it can be dynamically compiled and run using the moonloader. It’s as simple as require "moonscript" in order to have Lua understand how to load and run any MoonScript file.

Because it compiles right into Lua code, it is completely compatible with alternative Lua implementations like LuaJIT, and it is also compatible with all existing Lua code and libraries.

The command line tools also let you run MoonScript directly from the command line, like any first-class scripting language.

A comprehensive overview of the language can be found in the reference manual, the rest of this page serves as an overview of the language.

Overview

MoonScript provides a clean syntax using significant whitespace that avoids all the keyword noise typically seen in a Lua script. Below is a sample of some constructs found in the language.

exportmy_funcx=2323collection=height:32434hats:{"tophat","bball","bowler"}my_func=(a)->x+aprintmy_func100

It also adds table comprehensions, implicit return on functions, classes,inheritance, scope management statements import& export, and a convenient object creation statement called with.

importconcat,insertfromtabledouble_args=(...)->[x*2forxin*{...}]tuples=[{k,v}fork,vinipairsmy_table]

It can be loaded directly from a Lua script without an intermediate compile step. It even knows how to tell you where errors occurred in the original file when they happen.

Installation

Installing with LuaRocks

If you're on Windows, then install the Windows binaries, otherwise the easiest way to install is to use LuaRocks.

LuaRocks can be obtained here or from your package manager.

After it is installed, run the following in a terminal:

$ luarocks install moonscript

This will provide the moon and moonc executables along with themoonscript and moon Lua module.

Windows Binaries

Precompiled Windows binaries are available to avoid the trouble of compiling:moonscript.zip

Extract the contents into your PATH. You can also use the includedmoonscript.dll to require the module in.

This version has been compiled against Lua 5.1.

Optional

If you're on Linux and use watch mode (which compiles .moon files to .lua files as they are changed) you can installlinotify to use inotify instead of polling.

Source

The source code to the project lives on GitHub:
https://github.com/leafo/moonscript

Issues with the tool can be reported on the issue tracker:
https://github.com/leafo/moonscript/issues

The latest development version can be installed with the dev rockspec:

$ luarocks install \
    https://luarocks.org/manifests/leafo/moonscript-dev-1.rockspec

Dependencies

In addition to Lua 5.1 or 5.2, the following Lua modules are required to run the compiler and associated tools:

All of the required ones can be retrieved automatically using theLuaRocks installation.

Learning

Extras & Addons

Editor Support

Vim syntax and indent:
https://github.com/leafo/moonscript-vim

Sublime Text (and Textmate) syntax and indent:
https://github.com/leafo/moonscript-tmbundle

Tools

Online Compiler:
http://moonscript.org/compiler/

Overview of Differences & Highlights

A more detailed overview of the syntax can be found in thereference manual.

• Whitespace sensitive blocks defined by indenting
• All variable declarations are local by default
• export keyword to declare global variables, import keyword to make local copies of values from a table
• Parentheses are optional for function calls, similar to Ruby
• Fat arrow, =>, can be used to create a function with a self argument
• @ can be prefixed in front of a name to refer to that name in self
• ! operator can be used to call a function with no arguments
• Implicit return on functions based on the type of last statement
• : is used to separate key and value in table literals instead of =
• Newlines can be used as table literal entry delimiters in addition to ,
• \ is used to call a method on an object instead of :
• +=, -=, /=, *=, %=, ..= operators
• != is an alias for ~=
• Table comprehensions, with convenient slicing and iterator syntax
• Lines can be decorated with for loops and if statements at the end of the line
• If statements can be used as expressions
• Class system with inheritance based on metatable’s __index property
• Constructor arguments can begin with @ to cause them to automatically be assigned to the object
• Magic super function which maps to super class method of same name in a class method
• with statement lets you access anonymous object with short syntax

About

The syntax of MoonScript has been heavily inspired by the syntax ofCoffeeScript. MoonScript is CoffeeScript for Lua.

MoonScript would not have been possible without the excellent toolLPeg for parsing.

Changelog

• 0.5.0— September 25 2015
• 0.4.0— December 6 2015
• 0.3.2— June 1 2015
• 0.3.1— March 7 2015
• 0.3.0— February 28 2015
• 0.2.6— June 19 2014
• 0.2.5— March 5 2014
• 0.2.4— July 2 2013
• 0.2.3-2 — Jan 29 2013, Fixed bug with moonloader not loading anything
• 0.2.3— Jan 24 2013
• 0.2.2— Nov 04 2012
• 0.2.0— Dec 11 2011

In 1962, NASA launched the Mariner 2 probe past Venus, marking the first successful planetary flyby for the agency. It was done with an incredibly primitive computer that hardly fits the bill of anything we'd recognize today.

Each instrument worked on a tape loop, and the computer on board would run a sequence of commands based on an internal clock. It wasn't very sophisticated or easy to control. All input came from ground control, which could merely activate it to run certain pre-programmed sequences. (In fact, the lack of control led to the Mariner 1 craft's destruction when it failed to clock correctly.)

Later Mariner missions, which explored Mercury, Venus and Mars, were equipped with a very, very limited computer that was paired with the sequencer clocks. For instance, Mariner 8 could store data and run slightly more complex commands by kicking on sequencers in a cycle. Still, it only had about 100 commands it could understand based on a 512-word "vocabulary."

Chris Jones, a chief engineer at NASA's Jet Propulsion Laboratory, told Motherboard it was similar to a diagnostic check. While making flight adjustments, for instance, it would check the spacecraft's orientation and position against what the sequencers were programmed to do, "and if they don't match abort, the burn and try again."

"It's brand new science"

The 1970s brought immense change to NASA. In 1971, the central processing unit became commercially available. It allowed something that computers had struggled with before: handling multiple commands at once. And it let NASA enter a golden era of exploration, one that continues today with the Voyager probes. These probes, launched in 1977, visited the outer solar system and are now on the fastest trajectories of any craft as they head into interstellar space. They also have some of the first CPUs ever used by NASA, enabling the agency to move beyond the sequencer.

It enables the agency to re-program the craft as needed using primitive assembly languages like Cobol, Fortran, and Algol. The ability to reprogram the craft helped it move beyond the outer solar system and toward measuring particles in interstellar space.

"It's brand new science," Jone said. "It's never been seen before, so the Voyager team wants to extend that as long as they can."

In the 1970s, NASA launched four crafts that would head to the outer planets, and eventually leave the solar system. Pioneers 10 and 11 left Earth in 1972, and Voyagers 1 and 2 in 1977. The five-year gap between Pioneer and Voyager, while small, made all the difference in the world as computing advanced at a fast clip.

The Pioneer-10 spacecraft in 1972. Image: NASA Ames/Wikimedia Commons

The Pioneer crafts still worked with the sequencer-and-computer architecture, meaning they could only run the commands on board set by NASA Ames on the ground. The computer itself wasn't rewriteable and still mostly steered the craft toward certain commands, like analyzing for cosmic rays and micro-meteors or tracking solar winds, along with returning the first close images of Jupiter and Saturn.

The Voyager craft, on the other hand, were the first fly-by missions to have computers on them as we think of them today—albeit one only slightly less primitive than a Commodore 64. They could handle multiple commands, turn certain instruments on and off, and be fully reprogrammed. And, indeed, the need for programming and reprogramming the craft's onboard computers (even with their limited capabilities compared to today's machines) has been persistent in the 40 years they've been in space, flying out to distant stars.

The twin Voyager probes actually had three computers each, which are still functioning today—one for flight control, one for positioning of the craft, and another for the science payload, which is what transmits information on interstellar particles back to Earth. We'll be mostly concentrating on the science computer, called the Computer Command System, here.

The ability to write and rewrite code on the ground and reprogram the craft proved essential to Voyager's success

Jones was brought into the Voyager program at the beginning in 1973, when both Pioneer probes were already en route to Jupiter and Saturn. Jones previously worked on the later Mariner missions and understood their basic architecture, which was similar to Voyager's.

At that time, NASA was thinking even bigger. It wanted to do something no other agency had done: land an object on Mars. (The Soviets tried with Mars 2 and 3, both of which got there but neither of which were successful.) The Viking program was vital in our understanding of the red planet—and it also boasted the first CPU-based computer on-board any NASA probe which helped it perform some of the first hunts for microbial life on another planet.

Read More: How Viking 1 Won the Martian Space Race

That system was put inside the hardware of the Mariner probe architecture, creating a machine that could understand 4,096 words instead of just a few, one that was fully rewriteable, and offered control of all of the instruments on board on an ongoing basis, meaning that for some problems, there were workarounds in the case of instrument errors. (And there were plenty.)

When the Voyagers were initially launched in 1977, their design was clear: it was a four-year mission to get to Saturn. Anything after that would need to be greenlit by the powers-that-be at NASA. Jones and his team helped position Voyager 2 toward Uranus after its 1981 Saturn encounter. The ability to write and rewrite code on the ground and reprogram the craft proved essential to its success. This can be especially challenging when trying to reach a bus-sized craft that's so far away it takes 17 hours for communication to reach it, even though the signals are blazing toward it at the speed of light.

"After we'd flown the two missions to Saturn, we were pretty good at adding new data modes and working around problems," Jones said. "We became more conscious of what it could and couldn't do, and we took advantage of that in the Uranus and Neptune flybys."

At least twice, the Voyager team sent commands to the crafts that fixed errors. In 1978, Voyager 1 had to be reprogrammed to free up three instruments stuck in place by a combination of hardware and stubborn software. The mission was saved from failure by the computer inside. The lessons on fixing the scan platform helped recover Voyager 2 after a camera became misaligned in 1981, which could have jeopardized the science of the craft during its Uranus encounter (and thus, its Neptune encounter too).

Artist concept of Voyager in flight. Image: NASA/JPL/Wikimedia Commons

40 years later this September, and still equipped with those computers armed with (very) primitive microprocessors, the Voyager crafts are headed in separate directions out of the solar system. Both are in interstellar space, beyond the energy fields (but not the gravity) of the Sun. Both, unlike the Pioneer missions, are still fully in contact with NASA and performing essential science. Pioneer 10 lost contact with Earth in 2003 and Pioneer 11 in 1995, both crafts having expended their available energy beyond the ability to communicate back to Earth — in effect, because they weren't reprogrammable.

That requires a team of software engineers to continually reprogram the craft as it moves through unknown territory.

There aren't that many people left from the early days of Voyager, and the languages used to program it aren't widely taught today. The assembly languages it uses are now only found in remote areas of system architecture in computing, rather than in day-to-day programming.

This means that the Voyager team has to engage in retro-computing in order to get the craft to work continually and return science through at least the next half-decade, when the radiothermal gradiant power source may be too weak to keep the instruments online. They also have to troubleshoot the quirks that come with operating a 40-year-old computer that runs at 4 megahertz and is powered by a rapidly decaying radioactive "battery."

"They've learned some of the shortcuts and tricks that the original programmers used," Jones said. "It can turn a good day into a bad day if you stumble across one you didn't know."

The computing hardware used on the craft has long been supplanted by spaceships that use solid state memory instead of magnetic tape, and process information at gigahertz rather than low megahertz speeds. The Voyager crafts may be old, primitive, and at time clunky to work with, but above it all they endure.

And it's all thanks to a computer slower than some of the early desktop personal computers.

Subscribe to Science Solved It , Motherboard's new show about the greatest mysteries that were solved by science.

README.md

 mkdir lib;

PREAREIN_G_MODEL=

INPUT_IMAGE=

# -1 for cpu
GPU=0# "AtoB" or "BtoA"
DIRECTION="BtoA"

Introduction

Oliver Roeder runs a column on FiveThirtyEight called “The Riddler,” where he proposes logical and mathematical puzzles for readers to solve. On February 3rd of this year, he posted in Riddler Classic the problem, “Can You Rule Riddler Nation?” Here is the description:

In a distant, war-torn land, there are 10 castles. There are two warlords: you and your archenemy. Each castle has its own strategic value for a would-be conqueror. Specifically, the castles are worth 1, 2, 3, …, 9, and 10 victory points. You and your enemy each have 100 soldiers to distribute, any way you like, to fight at any of the 10 castles. Whoever sends more soldiers to a given castle conquers that castle and wins its victory points. If you each send the same number of troops, you split the points. You don’t know what distribution of forces your enemy has chosen until the battles begin. Whoever wins the most points wins the war.

Submit a plan distributing your 100 soldiers among the 10 castles. Once I receive all your battle plans, I’ll adjudicate all the possible one-on-one matchups. Whoever wins the most wars wins the battle royale and is crowned king or queen of Riddler Nation!

My solution was (1, 1, 1, 1, 1, 1, 23, 23, 24, 24), with this rationale:

Their are 55 possible points so one must win 28 to win. I will send 1 to each in case one person does not send any to a base. Then I will divide the remaining 90 among 7, 8, 9, 10 which is all that is nescecary to reach 28. 90/4 is 22 with 2 remainder. This will be put on the 9 and 10 to go after the most important targets.

The winning solution, by Cyrus Hettle, was (3, 5, 8, 10, 13, 1, 26, 30, 2, 2), and he won 84% of his battles, an excellent showing. In general, the winning strategies focused more on capturing lots of low-value castles than a few high-value ones, as my strategy did.

My strategy lost to the winning strategy, as the code below shows:

import pandas as pd
from pandas import Series, DataFrame
import numpy as np

def gen_strat(obj):
    """Generating a Series consisting of castle troop assignments

    args:
        obj: An iterable (NumPy array, Series, list, tuple, etc.) containing troop assignment

    return:
        Series; The generated strategy
    """

    srs = Series(obj).apply(int)
    if (len(srs) != 10) or (srs.sum() != 100) or ((srs < 0).values.any()):
        raise ValueError("A valid strategy consists of ten integers between 0 and 100 that sum to 100.")
    srs.index = pd.Index(np.arange(10) + 1)

    return srs

def check_strat(strat):
    """Checks that strat is a valid strategy, like that created by gen_strat

    args:
        strat: Series; An object to check

    return:
        bool; True if valid
    """

    if str(type(strat)) == "<class 'pandas.core.series.Series'>":
        # All the other conditions that must hold
        if len(strat) == 10 and strat.sum() == 100 and (strat >= 0).values.all() and \
           (strat == strat.apply(int)).values.all() and (strat.index.values == (np.arange(10) + 1)).all():
                return True

    return False

def battle_strat(strat1, strat2):
    """Determine which strategy between two wins

    args:
        strat1: Series; generated by gen_strat (or obeying its rules)
        strat2: Series; generated by gen_strat (or obeying its rules)

    return:
        dict; with the following fields:
            * "winner": int; 1 for strat1, -1 for strat2, 0 for tie
            * "pts": list; Item 0 is strat1's points, Item1 strat2's points
    """
    if not check_strat(strat1) or not check_strat(strat2):
        raise ValueError("Both strat1 and strat2 must be valid strategies")

    castle_margins = strat1 - strat2
    win1 = (castle_margins > 0).apply(int)
    win2 = (castle_margins < 0).apply(int)
    tie = (castle_margins == 0).apply(int)
    castle_points = Series(strat1.index, index=strat1.index)
    tie_pts = (tie * castle_points).sum() / 2
    pts1 = (win1 * castle_points).sum() + tie_pts
    pts2 = (win2 * castle_points).sum() + tie_pts
    pts_list = [pts1, pts2]

    if pts1 > pts2:
        return {"winner": 1, "pts": pts_list}
    elif pts1 < pts2:
        return {"winner": -1, "pts": pts_list}
    else:
        return {"winner": 0, "pts": pts_list}

myStrat  = gen_strat((1, 1, 1,  1,  1, 1, 23, 23, 24, 24))
winStrat = gen_strat((3, 5, 8, 10, 13, 1, 26, 30,  2,  2))

battle_strat(myStrat, winStrat)

{'pts': [22.0, 33.0], 'winner': -1}

You can see that my strategy went top-heavy, while the strategies that did best were bottom-heavy; they went for castles worth little and taking just enough point-heavy castles to win the tournament.

(Troops sent to castles, with winning distribution shown; credit to Oliver Roeder at FiveThirtyEight.)

This was my favorite of the Riddler challenges (I rarely submit to them but sometimes try to think of a solution), and I’m thrilled to see Mr. Roeder offer another one, but this time participants have the data for how others played, and Mr. Roeder encourages the players to use this data.

Challenge accepted, good sir.

Colonel Blotto Game

I read the first instantiation of the game while I was riding the train home from the U. of U., and I realized that, from a game-theoretic perspective, the game must have an optimal strategy (most likely a mixed strategy, where players randomly pick their moves; for example, the solution to rock-paper-scissors is a mixed strategy where a player randomly picks rock, paper, or scissors one third of the time for each), and I started thinking about how to find it, only to realize that the space of possible strategies is huge; in fact, there are 42,634,215,112,710 possible troop allocations. No way a computer is going to solve that. I tried to think of approximation methods that could possibly solve this problem, but my research turned up nothing. So I went off intuition (which didn’t work out well).

Sure enough, these problems are far from new, as Mr. Roeder discussed in the follow-up. They’re known as Colonel Blotto games.

It’s easy to show (TM) that these games have a Nash equilibrium and thus a solution, an optimal strategy. In fact, this pops straight out from Nash’s theorem; the only conditions required to apply his theorem are that there be two players (and there are), and that there be a finite number of potential strategies (which there is, although that finite number is in the trillions).

The hard part is finding the optimal strategy. In fact, that part is really hard. You can imagine why.

In some versions of the game, solutions can be found with analytic techniques. This was done for the version of the game proposed in a Rand Corporation paper in 1950 that is responsible for the name “Colonel Blotto”. The solution was found in 2006 by Brian Roberson, after 56 years of being unsolved. I doubt that the Battle for Riddler Nation is a game with a clean solution. It will likely require applying computational techniques that find approximate solutions rather than truly optimal ones.

Michael Wittman, a PhD candidate at MIT, wrote a paper applying agent-based models to the Colonel Blotto game, even providing some of the Python code he used. It’s an interesting read and I like the idea. His paper focuses on emulating round-robin tournaments like the Battle for Riddler Nation, where people create strategies, the researcher pits them against one-another, and sees which comes out on top. He also wanted to explore the characteristics of winning strategies.

I think that the agent-based approach could be used for finding quasi-optimal strategies, and I would like to try it here.

Agent-Based Modelling

Agent-based modelling is a computational approach for exploring complex systems by creating the agents that make up the system, putting them together, simulating their individual behaviors, allowing them to interact with one-another, and seeing what characteristics emerge in the larger system. For example, people apply agent-based models to traffic by creating a road network, creating simulated drivers, give them rules of behavior and desired destinations, and set them loose. It’s used in economics, sociology, biology, and other fields.

Agent-based modelling appeals for a number of reasons. First, it feels realistic to study a forest by simulating the trees and seeing what emerges. Second, the assumptions applied to agents seem more verifiable (and thus realistic) than making assumptions for the whole system. Third, agent-based models create a history that allows not only the end result but the dynamics leading up to that result to be studied. Fourth, they provide hope for solving problems for which an analytic solution is simply not possible, or confirming an analytic solution’s sufficiency. Fifth, they allow for investigating how sensitive a result is to initial conditions or assumptions. In a sense, they can help make sense of chaotic or fluid systems.

I’ve been interested in agent-based modelling for years. I would have loved to have done a thesis that applied agent-based models (ABMs) in an economic context. The appeal might stem from my life long love of video games and simulated systems, or just how cool I think it is to be creating petri-dish economies and societies on my computer. And I do believe that as computing power increases ABMs will be applied more for making forecasts and predictions, and that they are more realistic and general approaches to problems. Their results may be more trustworthy than those of analytic approaches, in my mind.

I’ve been planning to creating and applying ABMs for a while, and writing articles about them prior to the Battle for Riddler Nation. I want to use them to explore self-segregation of communities (see the seminal Schelling segregation model, which you can try without a computer), the emergence of two-party systems in winner-take-all elections (you will never convince me voting for Jill Stein or any Green Party nominee instead of the Democrat is a good idea), and maybe the development of technical analysis in financial markets. (So far I’ve found one book that might serve as a good starting point for studying ABMs, Agent-Based Models of the Economy; From Theories to Applications, by Boero et. al., which uses Python.)

Here, the idea is to create agents that always use some strategy for playing the Battle for Riddler Nation. Agents that lose their battles are dropped, while those that win spread. This approach emulates natural selection in nature; the “fit” reproduce, while the “unfit” do not, directing the species to the optimal design for their environment. Mr. Wittman applies such a technique in his paper, though I will make my own variations.

Unfortunately, the technique I just described does not find mixed strategies (that is, a strategy that randomly chooses a fixed strategy to apply). This is particularly bad in this context because one of the few things that is known with absolute certainty about Colonel Blotto games is that they have a mixed strategy, not a pure one. The hope is that whatever “stable” dynamics emerge will constitute a mixed strategy.

Here is an example. Suppose that we want to use agent-based modelling to solve rock-paper-scissors. We know that the optimal solution to rock-paper-scissors is to play rock, paper, and scissors randomly, choosing each one-third of the time in the long run. But the agents don’t play this way; each agent will either play either rock, paper, or scissors every game. So no one will be playing optimally, but hopefully one-third of all agents play pure-rock, one-third play pure-scissors, and one-third play pure-paper. Then the system, as a whole, represents an optimal mixed strategy.

I have no proof this is the case. This is just a hunch or hope. (Guess I have another project; see if this rock-paper-scissors dynamic as I described plays out).

Because the space of possible strategies in the Colonel Blotto-type games is so large, I would like the agents to explore the space. My idea is to allow mutations in the agents that get to reproduce. When an agent reproduces, the offspring has a chance to mutate, where the strategy they use is changed slightly. But as an agent lives longeer, the chance of producing a mutant offspring decreases, and those offspring that do mutate don’t mutate as much when they’re born from older agents. The rationale is that we should explore the sample space but we should bias it towards successful agents.

Again, I don’t know if this will work, but it’s worth a shot.

Mesa

While Wittman’s paper and Boero et. al‘s use Python and come with code examples, I want to use the Python library Mesa for creating an ABM. Mesa is a framework for creating and running ABMs and getting analytics from them, by David Masad and Jacqueline Kazil. It provides objects that can create agents, models, and visualizations. You can read Mr. Masad’s and Ms. Kazil’s paper, and watch their video presentations.

Mesa is still in development and there are models that it currently does not support, but the model that I will be using isn’t too fancy. It also is intended to integrate well with the rest of the scientific Python universe; pandasDataFrames and NumPy are supposedly supported by the package.

What Strategies to Use?

Mr. Wittman was interested in how to generate strategies for a round-robin tournament. Here, I’ve already been given a number of strategies, those used in the last Battle for Riddler Nation. My plan is to populate my world with agents that use the strategies players have already submitted.

Now, Mr. Roeder challenged players in the second round to use the results of the first round. I’m curious how people will process this. On the one hand, people may lean heavily towards emulating the top-performing strategies, in hopes ofad obtaining similar results. But then someone will come along who does just slightly better than the top-performers, or perhaps finds a flaw in that strategy and develops one that beats the best strategy and those similar to it. But then someone could realize this potentiality and develop a strategy that beats the strategy that beats what used to be the best strategy. And on and on and on…

Even with that in mind, I would rather just start with what I know, then hope that the mutations will account for these dynamics.

Here I load in the data file with the strategies submitted in the last Battle for Riddler Nation.

riddler_strats = pd.read_csv("castle-solutions.csv", encoding='latin-1').iloc[:, 0:10]    # Get only castle data

riddler_strats.head()

riddler_strats.columns = pd.Index(np.arange(10) + 1)
riddler_strats.head()

riddler_strats.shape

(1387, 10)

Some of the strategies people submitted were not valid; you can see that is in fact the case for the strategy with index 3 (it does not add up to 100). Here is a count of how many strategies are valid, followed by filtering the DataFrame for valid strategies.

riddler_strats.apply(lambda x: not check_strat(x), axis=1).sum()

38

riddler_strats = riddler_strats.loc[riddler_strats.apply(check_strat, axis=1), :]
riddler_strats.head()

And just for kicks, let’s see how my strategy performed against these.

myStrat_res = riddler_strats.apply(lambda x: battle_strat(myStrat, x)["winner"], axis=1)
myStrat_res.head()

0    1
1    1
2    1
4   -1
5   -1
dtype: int64

# Number of wins
(myStrat_res == 1).sum()

647

# Number of losses
(myStrat_res == -1).sum()

692

# Number of ties
(myStrat_res == 0).sum() - 1    # There is one auto-tie, against myself

9

647 / (692 + 647 + 9)

0.47997032640949555

With a win rate of 48%, my strategy didn’t do particularly well. But whatever. Water under the bridge now.

Anyway, just for the fun of it, I’m going to describe the ABM in a narrative.

The Great Wars for Riddler Nation

Riddler Nation has been split between 1,349 warlords. There are 4,047 provinces, with each province having ten castles, with values ranging from one to ten. Each warlord starts with three provinces. Wars take place and winners are determined as before.

Every year, every warlord wars with another warlord, chosen at random (Riddler Nation has teleportation magic, so every province is adjacent to every other province). Each warlord has his or her own war doctrine, and the warlord clings to it until death. When a warlord loses all the provinces under his control, the warlord dies, a pauper. But warlords only know war; they cannot manage their lands very well. The moment a warlord finds himself in charge of six provinces, he splits his domain, giving three provinces to his eldest son. The eldest son may have his own war doctrine, or may share his father’s, but the sons of the most successful warlords tend to stick with their fathers’ ways.

Let’s see how the lands of Riddler Nation evolve.

import mesa, mesa.time, mesa.datacollection
import random
import numpy as np
from numpy.random import poisson, uniform

class Warlord(mesa.Agent):
    """A warlord of Riddler Nation, who starts with a war doctrine, provinces, and fights other warlords"""
    def __init__(self, model, unique_id, doctrine, provinces, max_provinces, mutate_prob, mutate_amount):
        """Create a warlord

        args:
            model: mesa.Model; The model to which the agent belongs
            unique_id: int; Identifies the agent
            doctrine: Series; Describes the doctrine, and must be like one generated by gen_strat()
            provinces: int; The number of provinces the warlord starts with
            max_provinces: int; The maximum number of provinces a warlord can have before a split
            mutate_prob: float; A number between 0 and 1 that represents the probability of a mutation in offspring
            mutate_ammount: float; A number greater than 0 representing average number of mutations in doctrine
                                   of offspring

        return:
            Warlord
        """

        self.model = model

        if not check_strat(doctrine):
            raise ValueError("doctrine must be a valid strategy")

        self.doctrine = doctrine
        self.unique_id = int(unique_id)
        self.provinces = max(0, int(provinces))            # provinces at least 0
        self.max_provinces = max(2, int(max_provinces))    # max_provinces at least 2
        self.mutate_prob = max(min(mutate_prob, 1), 0)     # between 0 and 1
        self.mutate_amount = max(mutate_amount, 0)

        self.opponent = None    # In future, will be the id of this warlord's opponent

    def step(self):
        """In a step, find a new opponent to battle if not assigned"""

        if self.opponent is None:
            other = self.model.get_random_warlord()
            self.opponent = other
            other.opponent = self

    def advance(self):
        """Fight, and handle death or split of provinces"""

        if self.opponent is not None:
            # Resolve battle
            other = self.opponent
            res = battle_strat(self.doctrine, other.doctrine)["winner"]
            self.provinces += res
            other.provinces -= res
            # Clear opponents
            self.opponent = None
            other.opponent = None

        # Resolve possible death
        if self.provinces <= 0:
            self.model.schedule.remove(self)

        # If too many provinces, split, create new warlord
        if self.provinces >= self.max_provinces:
            son_doctrine = self.doctrine
            son_mutate_prob = self.mutate_prob
            son_mutate_amount = self.mutate_amount
            son_provinces = int(self.provinces / 2)

            # Is there a mutation?
            if uniform() < self.mutate_prob:
                # Yes! Apply the mutation
                son_mutate_prob = (self.mutate_prob + 1) / 2
                son_mutate_amount = self.mutate_amount * 2
                # Change doctrine
                mutations = min(poisson(self.mutate_amount), 100)
                for _ in range(mutations):
                    son_doctrine[random.choice(son_doctrine.index)] += 1                      # Add 1 to a random castle
                    son_doctrine[random.choice(son_doctrine[son_doctrine > 0].index)] -= 1    # Subtract 1 from a random
                                                                                              # castle that has at least
                                                                                              # one soldier
            # Create the son
            son = Warlord(model = self.model,
                          unique_id = self.model.next_id(),
                          doctrine = son_doctrine,
                          provinces = son_provinces,
                          max_provinces = self.max_provinces,
                          mutate_prob = son_mutate_prob,
                          mutate_amount = son_mutate_amount)
            self.model.schedule.add(son)

            # Changes to the warlord
            self.mutate_prob /= 2
            self.mutate_amount /= 2
            self.provinces = self.max_provinces - son_provinces

class RiddlerBattle(mesa.Model):
    """The Model that handles the simulation of the Battle for Riddler Nation"""
    def __init__(self, doctrines, N=None, max_provinces=6, mutate_prob=0.9, mutate_amount=10, debug=False):
        """Create an instance of the Battle for Riddler Nation

        args:
            doctrines: DataFrame; Contains all the doctrines for the warlords to create
            N: int; The number of warlords to create; if None, the number of rows of doctrines
            max_provinces: int; The maximum number of provinces each warlord can have
            mutate_prob: float; Each warlord's initial mutation probability
            mutate_amount: float; Each warlord's initial rate of number of mutations
            debug: bool; Debug mode

        return:
            RiddlerBattle
        """

        if N is None:
            N = doctrines.shape[0]

        if debug:
            strat = {"Doctrine": lambda w: w.doctrine.values, "Provinces": lambda w: w.provinces}
        else:
            strat = {"Doctrine": lambda w: w.doctrine.values}
        self.schedule = mesa.time.SimultaneousActivation(self)    # Battles are determined simultaneously
        self._max_id = 0    # The highest id currently used by the model
        self.dc = mesa.datacollection.DataCollector(agent_reporters=strat)    # Data collection

        self.create_warlords(doctrines, N, max_provinces, mutate_prob, mutate_amount)
        self._gen_warlord_list()

    def create_warlords(self, doctrines, N, max_provinces, mutate_prob, mutate_amount):
        """Populate the model with warlords

        args:
            doctrines: DataFrame; Contains all the doctrines for the warlords to create
            N: int; The number of warlords to create
            max_provinces: int; The maximum number of provinces each warlord can have
            mutate_prob: float; Each warlord's initial mutation probability
            mutate_amount: float; Each warlord's initial rate of number of mutations
        """

        i = 0
        init_provinces = int(max_provinces / 2)
        for _ in range(N):
            w = Warlord(model = self,
                        unique_id = self.next_id(),
                        doctrine = doctrines.iloc[i, :],
                        provinces = init_provinces,
                        max_provinces = max_provinces,
                        mutate_prob = mutate_prob,
                        mutate_amount = mutate_amount)
            self.schedule.add(w)

            i += 1
            if (i >= doctrines.shape[0]):
                i = 0    # We've run out of available doctrines, so start at the beginnning of the dataframe

    def _gen_warlord_list(self):
        """Generate a list of warlords used for assigning opponents"""
        self._warlord_list = [w for w in self.schedule.agents]

    def get_random_warlord(self):
        """Pull a random warlord without an opponent"""
        i = random.choice([i for i in range(len(self._warlord_list))])
        return self._warlord_list.pop(i)

    def next_id(self):
        """Get the next valid id for the model, and update the max id of the model

        return:
            int; Can be used as an id
        """

        self._max_id += 1
        return self._max_id

    def step(self):
        """Take a step"""
        self.dc.collect(self)
        self.schedule.step()
        self._gen_warlord_list()   # Reset the list of available warlords

    def run_model(self, steps):
        """Run the model

        args:
            steps: int; The number of steps to run the model
        """

        steps = int(steps)
        for _ in range(steps):
            self.step()

Having written the code defining the land of Riddler Nation, let’s run the model!

… first on a baby version, using only the first few strategies.

riddler_strats.head()    # The strategies being used

rb = RiddlerBattle(riddler_strats.head())

rb.run_model(100)    # 100 battles

strat_hist = rb.dc.get_agent_vars_dataframe()    # Get the doctrines used by agents
strat_hist.sort_index(inplace=True)
strat_hist.loc[(slice(0, 10)), :]    # See the first ten rounds

strat_hist.tail()

Looks like 100 steps was enough to reach stability. How does the new strategy perform?

strat_hist.head().applymap(lambda x: battle_strat(gen_strat(strat_hist.head().iloc[4, 0]), gen_strat(x))["winner"])

Not bad. We would expect an optimal strategy to tie or win at least half of the time. It will not win all the time, just like no one wins rock-paper-scissors all the time when both players play optimally.

Now let’s go to the full simulation.

%time rb_full = RiddlerBattle(riddler_strats)

CPU times: user 3.43 s, sys: 16 ms, total: 3.45 s
Wall time: 3.53 s

%time rb_full.run_model(100)    # A hundred years of warfare

CPU times: user 18min 29s, sys: 2.36 s, total: 18min 31s
Wall time: 18min 53s

warlords = rb_full.dc.get_agent_vars_dataframe()

warlords.loc[(99), :]

warlords.loc[(99), :].shape

(1659, 1)

battle_winners = DataFrame([w[1]["Doctrine"] for w in warlords.loc[(99), :].iterrows()], columns=np.arange(10) + 1,
                           index=warlords.loc[(99), :].index)
battle_winners.head()

def battle_strat_df(df1, df2):
    """Adjudicate all battles to see who wins

    args:
        df1: DataFrame; A data frame of warlords' strategies
        df2: DataFrame; A data frame of warlords' strategies

    return:
        DataFrame; Contains columns for Win, Tie, and Lose, the performance of the warlords of df1 against those
                   of df2
    """
    df1_mat = df1.values
    df2_mat = df2.values
    score_mat = np.ones(df2_mat.shape, dtype=np.int8)

    res = list()
    for i in range(df1_mat.shape[0]):
        vec = df1_mat[i, np.newaxis, :]
        margin = vec - df2_mat
        df1_castles = (margin > 0) * (np.arange(10) + 1)
        df2_castles = (margin < 0) * (np.arange(10) + 1)
        tie_castles = (margin == 0) * (np.arange(10) + 1)

        tie_scores = tie_castles.sum(axis=1) / 2
        df1_scores = df1_castles.sum(axis=1) + tie_scores
        df2_scores = df2_castles.sum(axis=1) + tie_scores
        res.append({"Win": (df1_scores > df2_scores).sum(),
                    "Tie": (df1_scores == df2_scores).sum(),
                    "Losses": (df1_scores < df2_scores).sum()})

    return DataFrame(res, index=df1.index)

results = battle_strat_df(battle_winners, riddler_strats)

results.sort_values("Win", ascending=False).head().join(battle_winners)

results.describe()

As a frame of reference, let’s see what the results for the original game were.

riddler_strat_results = battle_strat_df(riddler_strats, riddler_strats)

riddler_strat_results.sort_values("Win", ascending=False).head()

riddler_strat_results.describe()

riddler_strats.loc[1109, :]    # This is not the same as the winner according to official results

1      0
2      3
3      9
4      9
5      2
6     13
7     29
8     28
9      5
10     2
Name: 1109, dtype: int64

How do the warlords do against the winners of the simulated tournament?

warlord_results = battle_strat_df(battle_winners, battle_winners)

warlord_results.sort_values("Win", ascending=False).head().join(battle_winners)

warlord_results.describe()

Strangely, I got different results than Oliver Roeder, and I found a different winner, from row 1110 (or 1109 if you count from zero). I’m curious why that is.

That said, the best performer from the simulated warlords did not do as well as a human player, and uses the strategy (0, 0, 3, 7, 11, 20, 24, 29, 2, 4).

That said, among “optimally” playing warlords, the best had 1,309 victories, and used the strategy (3, 3, 13, 2, 4, 1, 27, 30, 10, 7). Now, if I were actually playing optimally, I would be playing a random strategy, so I would randomly pick one of the remaining warlords and use his strategy. But… I don’t want to do that. It’s a one-off game, so I’ll play the strategy that did best.

Conclusion

Do I expect to win the Battle for Riddler Nation? Am I using the best technique possible? Am I playing “optimally?”

By now, some readers may be screaming at me for begging the question: “What does it mean to play optimally?” Throughout this post, I’ve been referring to the game-theoretic notion of optimality. Here, this game is zero-sum, and it is fair; both players have access to the same set of strategies and determine their moves simultaneously, so neither player has an advantage. In this situation, optimal play means playing a strategy that guarantees a player, in the long run, will win at least as many games as her opponent no matter what strategy her opponent uses. In fact, this strategy does its job so well, she could announce her strategy to her opponent, and this guarantee will not be broken; she will still win at least as many games as she loses in the long run. (Not counting draws.) No strategy exists to break this guarantee. So game-theoretic optimal play is what’s known as minimax: it minimizes the maximum loss a player can suffer (in this case, all the way to zero).

How can this be the case? Often such strategies involve a player randomly picking her move (a mixed strategy), as opposed to always playing a certain move. Rock-paper-scissors is a perfect toy example. A pure strategy in this game would be “always play rock”, whereas the optimal mixed strategy is “play rock, paper, or scissors, picking one randomly with equal chance.” You can easily imagine that the optimal strategy can guarantee the player will win at least as many games as she loses, in this case. Who wins ultimately amounts to a coin flip.

But there’s a catch. True, rock-paper-scissors has an optimal strategy and if everyone played optimally then there would be no need for organizations such as the World RPS Society. Yet this society exists, and that’s because people rarely play optimally.

The problem is randomization. Even if people could find the optimal solution (and in the General Blotto game that’s quite unlikely), they can’t randomize well. There’s patterns they develop, or tips to their moves, or biases. In rock-paper-scissors, people rarely play the same strategy twice, let alone three times, and forget about four times (random strategies produce more long strings of moves than humans). They’re also prone to play whatever would have won the last round. The World RPS Society has a list of tricks to gain an edge on a human player.

People may also not fully understand what the optimal solution is. The classic example is a guessing game, where people pick numbers between 1 and 100, and the person who picks the answer that’s two-thirds of the average answer wins the game. According to game theory, the optimal answer is 0, found by multiplying two-thirds an infinite number of times. But in practice the winning number is never zero, nor even one. It’s much larger, since while most people appreciate they should not choose 66, nor choose 44, they rarely go beyond a few steps of the iteration, and may end up picking 30 on average, so the winner is the one who picked 20. (Mr. Roeder, in the same Riddler post as the one of the first Battle for Riddler Nation, actually proposed this game, but with the starting number being 1,000,000,000; the winning number, by Jeffery, was 196,352,731.)

Someone could win the Battle of Riddler Nation by better guessing what other players will do. Maybe 20% of players will do nothing different, 20% will play what won the last iteration, 20% will play what just barely beats the last iteration, 20% will play what beats the strategy that beats the strategy that barely won, and 20% will try to play in a style that tries to be game-theoretically optimal, and someone is able to guess this distribution, calculates what he should do, plays a strategy that’s good against humans, and beats 80% of participants.

Even if the odds of my winning the game are low, this was a fun project. Winning the game would be icing on the cake.

You know the saying about the best barber having the worst haircut? So it was for me. I've built many websites and webshops for our customers since the launch of Sabre, but my personal consulting website still remained on Wordpress. Well no more!

It took just 35 minutes to build an entire website!

The previous BIC site was built on Wordpress using a 59$ premium theme. I suspect the creator sold 5 or 6.000 copies of that theme, so the site wasn't exactly unique but nice enough. But reviewing the sourcefiles for the theme, its clear the creator spent 100 - 200 hours making it, so I think he would be quite thrilled to learn, that making such themes now takes as little as 35 minutes in Sabre. See for yourself, this is how I built the menu you see on bestinclass.dk:

Firstly you see me making two containers for both the logo and the menu items. Sabre automatically converts this to a bootstrap grid ensuring that it looks great on all devices. Then I simply open the content editor and insert my logo and type in the menu items. Interestingly enough, there are 3 different ways to make a menu, a big menu and a mega menu in Sabre, but I opted for the simplest version.

Life without templates

But the savvy webdeveloper will be wondering, if this quick performance boost won't come back to haunt me on all other pages since Im not using a template per say. The answer is no. If you make something, which you know will be used on other pages, like a header or a footer you simply give it a name by clicking the "Save Module" button (middle). Then afterwards, if you edit that element you'll see this button start glowing:

Clicking it, will scan every page and post on your website for this element (i.e. the header) and refresh it to match your pages. We believe this is the 2017 version of templates and that you shouldnt settle for any less. Once you've defined your modules, like header/footer, inserting them is very simple:

The little things

A major strength of Sabre is that we've compiled a list of all necessary styling properties you as a website/webshop manager need to build any kind of website. That means two things:

1. We let you do things in more ways than one
2. We cant cover it all in a blogpost

For now we'll just stick to the facts when it comes to migrating from Wordpress to Sabre CMS:

• From a premium theme costing 59$ => 35 minutes of work
• From a load time of 2.25 seconds => 700 milliseconds
• From a somewhat generic site => Completely personalized
• From Templates => Sabre!

Thanks for reading

Lau

Offentliggjort den 13. apr. 2014

Intro

We built Nile.js, a peer-to-peer live video streaming library designed to handle scaling. Our library uses WebTorrent, a distributed file delivery protocol inspired by BitTorrent and built with WebRTC. We chose WebTorrent as our means of broadcasting the stream because video streams can get progressively stronger as more peers join the stream. This also makes it a better fit than implementing typical WebRTC peer connections due to the approximately 10 to 20 connection limit per peer that WebTorrent has been able to mitigate.

How we use WebTorrent to stream

First, torrent files used by WebTorrent are immutable, making it less than ideal for live streaming. To emulate streaming, we take the video stream and split it into clips that are distributed to viewers through their own generated torrent files.

Once the WebTorrent client begins to seed, a magnet link (an identifier in the form of a string) is generated. We propagate the magnet to the viewing peers through a series of WebSocket and WebRTC connections. The magnet is first passed to a Node/Express server which is connected to a limited number of WebSockets. The server emits the magnet to the initial WebSocket connections which pass the magnet along through WebRTC peers in a linked list structure. After receiving the magnet, each viewer client will begin to leech (download) and seed (upload) the stream.

Considerations

Because of the linked list structure of our WebSocket to WebRTC connection, if a peer disconnects, the chain would be broken. Our library is designed to automatically reconnect on service disruptions. The disconnected webRTC peer will open a WebSocket connection with server, where it will be reassigned to a new WebSocket peer.

If there are only a few peers viewing the stream, initial upload speed is highly dependent on the broadcaster’s bandwidth and network speed, creating a major bottleneck. To offset this poor initial performance, we allowed the broadcaster enough time to send each peer enough data to view without missing a second. From this experience, we further acknowledge the effectiveness of combining peer-to-peer delivery with a CDN, a la Peer5, to handle the spectrum of low to high numbers of viewers.

Problems

As we rack up the number WebTorrent seeds on the broadcaster side, there seems to be a client side memory leak which eventually causes the browser to crash. We’ve checked the CPU and memory usage but have found no error messages or anomalies.

Unsuccessful approaches that we have taken to solve the issue:

• Server side seeding in place of client side seeding
• Seeding with Web Workers
• Seeding in iframes
• Destroying WebTorrent client on each new seed
• Destroying WebTorrent file on each new seed

One solution that we have found, is browser specific. If using Chrome Canary to test the library, it seems to run indefinitely. If anyone in the open source community is able to provide some insight, please make a pull request on our GitHub repo.

Conclusion

As a proof of concept, we would like other developers to try our library and leave feedback on how we can improve the project or even build upon it. Come check us out at our website, GitHub and npm.

Nile.js is Derek Miranda, Kevin Qiu, and Justin Pierson.

(Submitted on 27 Apr 2017)

Abstract: Artificial intelligence methods have often been applied to perform specific functions or tasks in the cyber-defense realm. However, as adversary methods become more complex and difficult to divine, piecemeal efforts to understand cyber-attacks, and malware-based attacks in particular, are not providing sufficient means for malware analysts to understand the past, present and future characteristics of malware.
In this paper, we present the Malware Analysis and Attributed using Genetic Information (MAAGI) system. The underlying idea behind the MAAGI system is that there are strong similarities between malware behavior and biological organism behavior, and applying biologically inspired methods to corpora of malware can help analysts better understand the ecosystem of malware attacks. Due to the sophistication of the malware and the analysis, the MAAGI system relies heavily on artificial intelligence techniques to provide this capability. It has already yielded promising results over its development life, and will hopefully inspire more integration between the artificial intelligence and cyber--defense communities.

Priced at $1,999, the 7980XE is clearly not a chip you'd see in an average desktop. Instead, it's more of a statement from Intel. It beats out AMD's 16-core Threadripper CPU, which was slated to be that company's most powerful consumer processor for 2017. And it gives Intel yet another way to satisfy the demands of power-hungry users who might want to do things like play games in 4K while broadcasting them in HD over Twitch. And as if its massive core count wasn't enough, the i9-7980XE is also the first Intel consumer chip that packs in over a teraflop worth of computing power.

If 18 cores is a bit too rich for you, Intel also has other Core i9 Extreme Edition chips in 10, 12, 14 and 16-core variants. Perhaps the best news for hardware geeks: the 10 core i9-7900X will retail for $999, a significant discount from last year's version.

All of the i9 chips feature base clock speeds of 3.3GHz, reaching up to 4.3GHz dual-core speeds with Turbo Boost 2.0 and 4.5GHz with Turbo Boost 3.0. And speaking of Turbo Boost 3.0, its performance has also been improved in the new Extreme Edition chips to increase both single and dual-core speeds. Rounding out the X-Series family are the quad-core i5-7640X and i7 models in 4, 6 and 8-core models.

While it might all seem like overkill, Intel says its Core i9 lineup was driven by the surprising demand for last year's 10-core chip. "Broadwell-E was kind of an experiment," an Intel rep said. "It sold... Proving that our enthusiast community will go after the best of the best... Yes we're adding higher core count, but we're also introducing lower core counts. Scalability on both ends are what we went after."

As you can imagine, stuffing more cores into a processor leads to some significant heat issues. For that reason, Intel developed its own liquid cooling solution, which will work across these new chips, as well as some previous generations. All of the new Core i9 processors, along with the 6 and 8-core i7 chips, feature scorching hot 140W thermal design points (TDPs), the maximum amount of power that they'll draw. That's the same as last year's 10-core CPU, but it's still well above the 91W TDP from Intel's more affordable i7-7700K.

Over the past few years, Intel's laptop chips have been far more interesting than its desktop CPUs. Partially, that's because the rise of ultraportables and convertible laptops have shifted its focus away from delivering as much computing power as possible, to offering a reasonable amount of processing power efficiently. The new Core i9 X-series processors might not be feasible for most consumers, but for the hardware geeks who treat their rigs like hot rods, they're a dream come true.

Clojure is a great programming language, but a recurring complaint one keeps hearing from developers hacking on Clojure code is that debugging can be unpleasant. First of all, I agree! Debugging Clojure code can be more daunting on average than, say, debugging Python code. This is mainly due to two reasons:

1. Clojure's Java legacy. Clojure is compiled to Java bytecode, which has some terminology and idiosyncracies Clojure programmers aren't always familiar with. These terms tend to pop up in stack traces and cause confusion (e.g.IFN).
2. Clojure - being a Lisp - has a certain code structure which is different from, say, a more common imperative coding style. Rather than being a sequence of statements, Clojure programs tend to involve long call chains of nested expressions. Where only part of an expression fails, it's often non-trivial to figure out why.

In this post I want to share some notes from my own experience debugging Clojure programs.

(defn foo[n](cond (> n40)(+ n20)(> n20)(- (first n)20):else0))

debugging.core=> (foo 24)
IllegalArgumentException Don't know how to create ISeq from: java.lang.Long
  clojure.lang.RT.seqFrom (RT.java:542)

debugging.core=> (pst)
IllegalArgumentException Don't know how to create ISeq from: java.lang.Long
  clojure.lang.RT.seqFrom (RT.java:542)
  clojure.lang.RT.seq (RT.java:523)
  clojure.lang.RT.first (RT.java:668)
  clojure.core/first--4339 (core.clj:55)
  clojure.core/first--4339 (core.clj:55)
  debugging.sample/foo (sample.clj:10)
  debugging.sample/foo (sample.clj:7)
  debugging.core/eval13715 (form-init6539101589609174055.clj:1)
  debugging.core/eval13715 (form-init6539101589609174055.clj:1)
  clojure.lang.Compiler.eval (Compiler.java:6927)
  clojure.lang.Compiler.eval (Compiler.java:6890)
  clojure.core/eval (core.clj:3105)

(> n20)(- (first n)20)

debugging.core=> (foo nil)

NullPointerException   clojure.lang.Numbers.ops (Numbers.java:1013)

debugging.core=> (pst)
NullPointerException
  clojure.lang.Numbers.ops (Numbers.java:1013)
  clojure.lang.Numbers.gt (Numbers.java:229)
  clojure.lang.Numbers.gt (Numbers.java:3864)
  debugging.sample/foo (sample.clj:9)
  debugging.sample/foo (sample.clj:7)
  debugging.core/eval14693 (form-init6539101589609174055.clj:1)
  debugging.core/eval14693 (form-init6539101589609174055.clj:1)
  clojure.lang.Compiler.eval (Compiler.java:6927)
  clojure.lang.Compiler.eval (Compiler.java:6890)
  clojure.core/eval (core.clj:3105)
  clojure.core/eval (core.clj:3101)
  clojure.main/repl/read-eval-print--7408/fn--7411 (main.clj:240)

(cond (> c40)(* (+ n20)(+ m30)))

(defn foo[n](trace-forms(cond (> n40)(+ n20)(> n20)(- (first n)20):else0)))

debugging.core=> (foo nil)
NullPointerException : No message attached to throwable java.lang.NullPointerException
  Form failed: (> n 40)
  Form failed: (if
 (> n 40)
 (+ n 20)
 (clojure.core/cond (> n 20) (- (first n) 20) :else 0))
  Form failed: (cond (> n 40) (+ n 20) (> n 20) (- (first n) 20) :else 0)
  clojure.lang.Numbers.ops (Numbers.java:1013)

(def ^:dynamic*verbose*false)(defmacro printfv[fmt&args]`(when *verbose*(printf~fmt~@args)))

(defmacro with-verbose[&body]`(binding [*verbose*true]~@body))

(defn factorial[n](printfv"factorial: %d%n"n)(if (< n1)1(* n(factorial(- n1)))))

debugging.core=> (factorial 6)
720

debugging.core=> (with-verbose (factorial 6))
factorial: 6
factorial: 5
factorial: 4
factorial: 3
factorial: 2
factorial: 1
factorial: 0
720

debugging.core=> (trace-vars factorial)
#'debugging.core/factorial

debugging.core=> (factorial 6)
TRACE t16315: (debugging.core/factorial 6)
TRACE t16316: | (debugging.core/factorial 5)
TRACE t16317: | | (debugging.core/factorial 4)
TRACE t16318: | | | (debugging.core/factorial 3)
TRACE t16319: | | | | (debugging.core/factorial 2)
TRACE t16320: | | | | | (debugging.core/factorial 1)
TRACE t16321: | | | | | | (debugging.core/factorial 0)
TRACE t16321: | | | | | | => 1
TRACE t16320: | | | | | => 1
TRACE t16319: | | | | => 2
TRACE t16318: | | | => 6
TRACE t16317: | | => 24
TRACE t16316: | => 120
TRACE t16315: => 720
720

(defn iseven?[n](if (= n0)true(isodd?(- n1))))(defn isodd?[n](if (= n0)false(iseven?(- n1))))

debugging.core=> (trace-vars iseven? isodd?)
#'debugging.core/isodd?
debugging.core=> (iseven? 7)
TRACE t16332: (debugging.core/iseven? 7)
TRACE t16333: | (debugging.core/isodd? 6)
TRACE t16334: | | (debugging.core/iseven? 5)
TRACE t16335: | | | (debugging.core/isodd? 4)
TRACE t16336: | | | | (debugging.core/iseven? 3)
TRACE t16337: | | | | | (debugging.core/isodd? 2)
TRACE t16338: | | | | | | (debugging.core/iseven? 1)
TRACE t16339: | | | | | | | (debugging.core/isodd? 0)
TRACE t16339: | | | | | | | => false
TRACE t16338: | | | | | | => false
TRACE t16337: | | | | | => false
TRACE t16336: | | | | => false
TRACE t16335: | | | => false
TRACE t16334: | | => false
TRACE t16333: | => false
TRACE t16332: => false
false

(cond (> 1020)(+ 1020)(> 2010)(- 2010):else200)

(defmacro condv[&clauses](when clauses(list'if(first clauses)(if (next clauses)`(do (println (str "condv "'~(first clauses)))~(second clauses))(throw(IllegalArgumentException."cond requires an even number of forms")))(cons 'condv(next (next clauses))))))

debugging.core=> (condv (> 10 20) (+ 10 20)
            #_=>        (> 20 10) (- 20 10)
            #_=>        :else 200)
condv (> 20 10)
10

SYSCALLS(2)               Linux Programmer's Manual              SYSCALLS(2)

NAME         top

       syscalls - Linux system calls

SYNOPSIS         top

       Linux system calls.

DESCRIPTION         top

       The system call is the fundamental interface between an application
       and the Linux kernel.System calls and library wrapper functions
       System calls are generally not invoked directly, but rather via
       wrapper functions in glibc (or perhaps some other library).  For
       details of direct invocation of a system call, see intro(2).  Often,
       but not always, the name of the wrapper function is the same as the
       name of the system call that it invokes.  For example, glibc contains
       a function truncate() which invokes the underlying "truncate" system
       call.

       Often the glibc wrapper function is quite thin, doing little work
       other than copying arguments to the right registers before invoking
       the system call, and then setting errno appropriately after the
       system call has returned.  (These are the same steps that are
       performed by syscall(2), which can be used to invoke system calls for
       which no wrapper function is provided.)  Note: system calls indicate
       a failure by returning a negative error number to the caller; when
       this happens, the wrapper function negates the returned error number
       (to make it positive), copies it to errno, and returns -1 to the
       caller of the wrapper.

       Sometimes, however, the wrapper function does some extra work before
       invoking the system call.  For example, nowadays there are (for
       reasons described below) two related system calls, truncate(2) andtruncate64(2), and the glibc truncate() wrapper function checks which
       of those system calls are provided by the kernel and determines which
       should be employed.System call list
       Below is a list of the Linux system calls.  In the list, the Kernel
       column indicates the kernel version for those system calls that were
       new in Linux 2.2, or have appeared since that kernel version.  Note
       the following points:

       *  Where no kernel version is indicated, the system call appeared in
          kernel 1.0 or earlier.

       *  Where a system call is marked "1.2" this means the system call
          probably appeared in a 1.1.x kernel version, and first appeared in
          a stable kernel with 1.2.  (Development of the 1.2 kernel was
          initiated from a branch of kernel 1.0.6 via the 1.1.x unstable
          kernel series.)

       *  Where a system call is marked "2.0" this means the system call
          probably appeared in a 1.3.x kernel version, and first appeared in
          a stable kernel with 2.0.  (Development of the 2.0 kernel was
          initiated from a branch of kernel 1.2.x, somewhere around 1.2.10,
          via the 1.3.x unstable kernel series.)

       *  Where a system call is marked "2.2" this means the system call
          probably appeared in a 2.1.x kernel version, and first appeared in
          a stable kernel with 2.2.0.  (Development of the 2.2 kernel was
          initiated from a branch of kernel 2.0.21 via the 2.1.x unstable
          kernel series.)

       *  Where a system call is marked "2.4" this means the system call
          probably appeared in a 2.3.x kernel version, and first appeared in
          a stable kernel with 2.4.0.  (Development of the 2.4 kernel was
          initiated from a branch of kernel 2.2.8 via the 2.3.x unstable
          kernel series.)

       *  Where a system call is marked "2.6" this means the system call
          probably appeared in a 2.5.x kernel version, and first appeared in
          a stable kernel with 2.6.0.  (Development of kernel 2.6 was
          initiated from a branch of kernel 2.4.15 via the 2.5.x unstable
          kernel series.)

       *  Starting with kernel 2.6.0, the development model changed, and new
          system calls may appear in each 2.6.x release.  In this case, the
          exact version number where the system call appeared is shown.
          This convention continues with the 3.x kernel series, which
          followed on from kernel 2.6.39, and the 4.x kernel series, which
          followed on from kernel 3.19.

       *  In some cases, a system call was added to a stable kernel series
          after it branched from the previous stable kernel series, and then
          backported into the earlier stable kernel series.  For example
          some system calls that appeared in 2.6.x were also backported into
          a 2.4.x release after 2.4.15.  When this is so, the version where
          the system call appeared in both of the major kernel series is
          listed.

       The list of system calls that are available as at kernel 4.11 (or in
       a few cases only on older kernels) is as follows:

       System call                Kernel        Notes────────────────────────────────────────────────────────────────────

       _llseek(2)                 1.2_newselect(2)              2.0_sysctl(2)                 2.0accept(2)                  2.0           See notes on socketcall(2)accept4(2)                 2.6.28access(2)                  1.0acct(2)                    1.0add_key(2)                 2.6.10adjtimex(2)                1.0alarm(2)                   1.0alloc_hugepages(2)         2.5.36        Removed in 2.5.44bdflush(2)                 1.2           Deprecated (does nothing)
                                                since 2.6bind(2)                    2.0           See notes on socketcall(2)bpf(2)                     3.18brk(2)                     1.0cacheflush(2)              1.2           Not on x86capget(2)                  2.2capset(2)                  2.2chdir(2)                   1.0chmod(2)                   1.0chown(2)                   2.2           See chown(2) for
                                                version detailschown32(2)                 2.4chroot(2)                  1.0clock_adjtime(2)           2.6.39clock_getres(2)            2.6clock_gettime(2)           2.6clock_nanosleep(2)         2.6clock_settime(2)           2.6clone(2)                   1.0close(2)                   1.0connect(2)                 2.0           See notes on socketcall(2)copy_file_range(2)         4.5creat(2)                   1.0create_module(2)           1.0           Removed in 2.6delete_module(2)           1.0dup(2)                     1.0dup2(2)                    1.0dup3(2)                    2.6.27epoll_create(2)            2.6epoll_create1(2)           2.6.27epoll_ctl(2)               2.6epoll_pwait(2)             2.6.19epoll_wait(2)              2.6eventfd(2)                 2.6.22eventfd2(2)                2.6.27execve(2)                  1.0execveat(2)                3.19exit(2)                    1.0exit_group(2)              2.6faccessat(2)               2.6.16fadvise64(2)               2.6fadvise64_64(2)            2.6fallocate(2)               2.6.23fanotify_init(2)           2.6.37fanotify_mark(2)           2.6.37fchdir(2)                  1.0fchmod(2)                  1.0fchmodat(2)                2.6.16fchown(2)                  1.0fchown32(2)                2.4fchownat(2)                2.6.16fcntl(2)                   1.0fcntl64(2)                 2.4fdatasync(2)               2.0fgetxattr(2)               2.6; 2.4.18finit_module(2)            3.8flistxattr(2)              2.6; 2.4.18flock(2)                   2.0fork(2)                    1.0free_hugepages(2)          2.5.36        Removed in 2.5.44fremovexattr(2)            2.6; 2.4.18fsetxattr(2)               2.6; 2.4.18fstat(2)                   1.0fstat64(2)                 2.4fstatat64(2)               2.6.16fstatfs(2)                 1.0fstatfs64(2)               2.6fsync(2)                   1.0ftruncate(2)               1.0ftruncate64(2)             2.4futex(2)                   2.6futimesat(2)               2.6.16get_kernel_syms(2)         1.0           Removed in 2.6get_mempolicy(2)           2.6.6get_robust_list(2)         2.6.17get_thread_area(2)         2.6getcpu(2)                  2.6.19getcwd(2)                  2.2getdents(2)                2.0getdents64(2)              2.4getegid(2)                 1.0getegid32(2)               2.4geteuid(2)                 1.0geteuid32(2)               2.4getgid(2)                  1.0getgid32(2)                2.4getgroups(2)               1.0getgroups32(2)             2.4getitimer(2)               1.0getpeername(2)             2.0           See notes on socketcall(2)getpagesize(2)             2.0           Not on x86getpgid(2)                 1.0getpgrp(2)                 1.0getpid(2)                  1.0getppid(2)                 1.0getpriority(2)             1.0getrandom(2)               3.17getresgid(2)               2.2getresgid32(2)             2.4getresuid(2)               2.2getresuid32(2)             2.4getrlimit(2)               1.0getrusage(2)               1.0getsid(2)                  2.0getsockname(2)             2.0           See notes on socketcall(2)getsockopt(2)              2.0           See notes on socketcall(2)gettid(2)                  2.4.11gettimeofday(2)            1.0getuid(2)                  1.0getuid32(2)                2.4getunwind(2)               2.4.8         ia64; deprecatedgetxattr(2)                2.6; 2.4.18init_module(2)             1.0inotify_add_watch(2)       2.6.13inotify_init(2)            2.6.13inotify_init1(2)           2.6.27inotify_rm_watch(2)        2.6.13io_cancel(2)               2.6io_destroy(2)              2.6io_getevents(2)            2.6io_setup(2)                2.6io_submit(2)               2.6ioctl(2)                   1.0ioperm(2)                  1.0iopl(2)                    1.0ioprio_get(2)              2.6.13ioprio_set(2)              2.6.13ipc(2)                     1.0kcmp(2)                    3.5kern_features(2)           3.7           Sparc64kexec_file_load(2)         3.17kexec_load(2)              2.6.13keyctl(2)                  2.6.10kill(2)                    1.0lchown(2)                  1.0           See chown(2) for
                                                version detailslchown32(2)                2.4lgetxattr(2)               2.6; 2.4.18link(2)                    1.0linkat(2)                  2.6.16listen(2)                  2.0           See notes on socketcall(2)listxattr(2)               2.6; 2.4.18llistxattr(2)              2.6; 2.4.18lookup_dcookie(2)          2.6lremovexattr(2)            2.6; 2.4.18lseek(2)                   1.0lsetxattr(2)               2.6; 2.4.18lstat(2)                   1.0lstat64(2)                 2.4madvise(2)                 2.4mbind(2)                   2.6.6membarrier(3)              3.17memfd_create(2)            3.17migrate_pages(2)           2.6.16mincore(2)                 2.4mkdir(2)                   1.0mkdirat(2)                 2.6.16mknod(2)                   1.0mknodat(2)                 2.6.16mlock(2)                   2.0mlock2(2)                  4.4mlockall(2)                2.0mmap(2)                    1.0mmap2(2)                   2.4modify_ldt(2)              1.0mount(2)                   1.0move_pages(2)              2.6.18mprotect(2)                1.0mq_getsetattr(2)           2.6.6mq_notify(2)               2.6.6mq_open(2)                 2.6.6mq_timedreceive(2)         2.6.6mq_timedsend(2)            2.6.6mq_unlink(2)               2.6.6mremap(2)                  2.0msgctl(2)                  2.0           See notes on ipc(2)msgget(2)                  2.0           See notes on ipc(2)msgrcv(2)                  2.0           See notes on ipc(2)msgsnd(2)                  2.0           See notes on ipc(2)msync(2)                   2.0munlock(2)                 2.0munlockall(2)              2.0munmap(2)                  1.0name_to_handle_at(2)       2.6.39nanosleep(2)               2.0nfsservctl(2)              2.2           Removed in 3.1nice(2)                    1.0oldfstat(2)                1.0oldlstat(2)                1.0oldolduname(2)             1.0oldstat(2)                 1.0olduname(2)                1.0open(2)                    1.0open_by_handle_at(2)       2.6.39openat(2)                  2.6.16pause(2)                   1.0pciconfig_iobase(2)        2.2.15; 2.4   Not on x86pciconfig_read(2)          2.0.26; 2.2   Not on x86pciconfig_write(2)         2.0.26; 2.2   Not on x86perf_event_open(2)         2.6.31        Was perf_counter_open() in
                                                2.6.31; renamed in 2.6.32personality(2)             1.2perfctr(2)                 2.2           Sparc; removed in 2.6.34perfmonctl(2)              2.4           ia64pipe(2)                    1.0pipe2(2)                   2.6.27pivot_root(2)              2.4pkey_alloc(2)              4.8pkey_free(2)               4.8pkey_mprotect(2)           4.8poll(2)                    2.0.36; 2.2ppc_rtas(2)                2.6.2         PowerPC onlyppc_swapcontext(2)         2.6.3         PowerPC onlyppoll(2)                   2.6.16prctl(2)                   2.2pread64(2)                               Added as "pread" in 2.2;
                                                renamed "pread64" in 2.6preadv(2)                  2.6.30preadv2(2)                 4.6prlimit64(2)               2.6.36process_vm_readv(2)        3.2process_vm_writev(2)       3.2pselect6(2)                2.6.16ptrace(2)                  1.0pwrite64(2)                              Added as "pwrite" in 2.2;
                                                renamed "pwrite64" in 2.6pwritev(2)                 2.6.30pwritev2(2)                4.6query_module(2)            2.2           Removed in 2.6quotactl(2)                1.0read(2)                    1.0readahead(2)               2.4.13readdir(2)                 1.0readlink(2)                1.0readlinkat(2)              2.6.16readv(2)                   2.0reboot(2)                  1.0recv(2)                    2.0           See notes on socketcall(2)recvfrom(2)                2.0           See notes on socketcall(2)recvmsg(2)                 2.0           See notes on socketcall(2)recvmmsg(2)                2.6.33remap_file_pages(2)        2.6           Deprecated since 3.16removexattr(2)             2.6; 2.4.18rename(2)                  1.0renameat(2)                2.6.16renameat2(2)               3.15request_key(2)             2.6.10restart_syscall(2)         2.6rmdir(2)                   1.0rt_sigaction(2)            2.2rt_sigpending(2)           2.2rt_sigprocmask(2)          2.2rt_sigqueueinfo(2)         2.2rt_sigreturn(2)            2.2rt_sigsuspend(2)           2.2rt_sigtimedwait(2)         2.2rt_tgsigqueueinfo(2)       2.6.31s390_runtime_instr(2)      3.7           s390 onlys390_pci_mmio_read(2)      3.19          s390 onlys390_pci_mmio_write(2)     3.19          s390 onlysched_get_priority_max(2)  2.0sched_get_priority_min(2)  2.0sched_getaffinity(2)       2.6sched_getattr(2)           3.14sched_getparam(2)          2.0sched_getscheduler(2)      2.0sched_rr_get_interval(2)   2.0sched_setaffinity(2)       2.6sched_setattr(2)           3.14sched_setparam(2)          2.0sched_setscheduler(2)      2.0sched_yield(2)             2.0seccomp(2)                 3.17select(2)                  1.0semctl(2)                  2.0           See notes on ipc(2)semget(2)                  2.0           See notes on ipc(2)semop(2)                   2.0           See notes on ipc(2)semtimedop(2)              2.6; 2.4.22send(2)                    2.0           See notes on socketcall(2)sendfile(2)                2.2sendfile64(2)              2.6; 2.4.19sendmmsg(2)                3.0sendmsg(2)                 2.0           See notes on socketcall(2)sendto(2)                  2.0           See notes on socketcall(2)set_mempolicy(2)           2.6.6set_robust_list(2)         2.6.17set_thread_area(2)         2.6set_tid_address(2)         2.6setdomainname(2)           1.0setfsgid(2)                1.2setfsgid32(2)              2.4setfsuid(2)                1.2setfsuid32(2)              2.4setgid(2)                  1.0setgid32(2)                2.4setgroups(2)               1.0setgroups32(2)             2.4sethostname(2)             1.0setitimer(2)               1.0setns(2)                   3.0setpgid(2)                 1.0setpriority(2)             1.0setregid(2)                1.0setregid32(2)              2.4setresgid(2)               2.2setresgid32(2)             2.4setresuid(2)               2.2setresuid32(2)             2.4setreuid(2)                1.0setreuid32(2)              2.4setrlimit(2)               1.0setsid(2)                  1.0setsockopt(2)              2.0           See notes on socketcall(2)settimeofday(2)            1.0setuid(2)                  1.0setuid32(2)                2.4setup(2)                   1.0           Removed in 2.2setxattr(2)                2.6; 2.4.18sgetmask(2)                1.0shmat(2)                   2.0           See notes on ipc(2)shmctl(2)                  2.0           See notes on ipc(2)shmdt(2)                   2.0           See notes on ipc(2)shmget(2)                  2.0           See notes on ipc(2)shutdown(2)                2.0           See notes on socketcall(2)sigaction(2)               1.0sigaltstack(2)             2.2signal(2)                  1.0signalfd(2)                2.6.22signalfd4(2)               2.6.27sigpending(2)              1.0sigprocmask(2)             1.0sigreturn(2)               1.0sigsuspend(2)              1.0socket(2)                  2.0           See notes on socketcall(2)socketcall(2)              1.0socketpair(2)              2.0           See notes on socketcall(2)splice(2)                  2.6.17spu_create(2)              2.6.16        PowerPC onlyspu_run(2)                 2.6.16        PowerPC onlyssetmask(2)                1.0stat(2)                    1.0stat64(2)                  2.4statfs(2)                  1.0statfs64(2)                2.6statx(2)                   4.11stime(2)                   1.0subpage_prot(2)            2.6.25        PowerPC onlyswapoff(2)                 1.0swapon(2)                  1.0symlink(2)                 1.0symlinkat(2)               2.6.16sync(2)                    1.0sync_file_range(2)         2.6.17sync_file_range2(2)        2.6.22syncfs(2)                  2.6.39sysfs(2)                   1.2sysinfo(2)                 1.0syslog(2)                  1.0tee(2)                     2.6.17tgkill(2)                  2.6time(2)                    1.0timer_create(2)            2.6timer_delete(2)            2.6timer_getoverrun(2)        2.6timer_gettime(2)           2.6timer_settime(2)           2.6timerfd_create(2)          2.6.25timerfd_gettime(2)         2.6.25timerfd_settime(2)         2.6.25times(2)                   1.0tkill(2)                   2.6; 2.4.22truncate(2)                1.0truncate64(2)              2.4ugetrlimit(2)              2.4umask(2)                   1.0umount(2)                  1.0umount2(2)                 2.2uname(2)                   1.0unlink(2)                  1.0unlinkat(2)                2.6.16unshare(2)                 2.6.16uselib(2)                  1.0ustat(2)                   1.0userfaultfd(2)             4.3utime(2)                   1.0utimensat(2)               2.6.22utimes(2)                  2.2utrap_install(2)           2.2           Sparc onlyvfork(2)                   2.2vhangup(2)                 1.0vm86old(2)                 1.0           Was "vm86"; renamed in
                                                2.0.28/2.2vm86(2)                    2.0.28; 2.2vmsplice(2)                2.6.17wait4(2)                   1.0waitid(2)                  2.6.10waitpid(2)                 1.0write(2)                   1.0writev(2)                  2.0

       On many platforms, including x86-32, socket calls are all multiplexed
       (via glibc wrapper functions)  through  socketcall(2)  and  similarly
       System V IPC calls are multiplexed through ipc(2).

       Although  slots  are  reserved for them in the system call table, the
       following system calls are not implemented in  the  standard  kernel:
       afs_syscall(2),  break(2),  ftime(2),  getpmsg(2),  gtty(2), idle(2),lock(2),   madvise1(2),   mpx(2),   phys(2),   prof(2),    profil(2),putpmsg(2),   security(2),   stty(2),   tuxcall(2),   ulimit(2),  andvserver(2)   (see   also   unimplemented(2)).    However,   ftime(3),profil(3),  and  ulimit(3)  exist  as library routines.  The slot forphys(2) is in use since kernel 2.1.116 for  umount(2);  phys(2)  will
       never  be  implemented.   The getpmsg(2) and putpmsg(2) calls are for
       kernels patched to support STREAMS, and may never be in the  standard
       kernel.

       There  was  briefly  set_zone_reclaim(2),  added in Linux 2.6.13, and
       removed in 2.6.16; this system  call  was  never  available  to  user
       space.

NOTES         top

       Roughly speaking, the code belonging to the system call with number
       __NR_xxx defined in /usr/include/asm/unistd.h can be found in the
       Linux kernel source in the routine sys_xxx().  (The dispatch table
       for i386 can be found in /usr/src/linux/arch/i386/kernel/entry.S.)
       There are many exceptions, however, mostly because older system calls
       were superseded by newer ones, and this has been treated somewhat
       unsystematically.  On platforms with proprietary operating-system
       emulation, such as parisc, sparc, sparc64, and alpha, there are many
       additional system calls; mips64 also contains a full set of 32-bit
       system calls.

       Over time, changes to the interfaces of some system calls have been
       necessary.  One reason for such changes was the need to increase the
       size of structures or scalar values passed to the system call.
       Because of these changes, certain architectures (notably,
       longstanding 32-bit architectures such as i386) now have various
       groups of related system calls (e.g., truncate(2) and truncate64(2))
       which perform similar tasks, but which vary in details such as the
       size of their arguments.  (As noted earlier, applications are
       generally unaware of this: the glibc wrapper functions do some work
       to ensure that the right system call is invoked, and that ABI
       compatibility is preserved for old binaries.)  Examples of systems
       calls that exist in multiple versions are the following:

       *  By now there are three different versions of stat(2): sys_stat()
          (slot __NR_oldstat), sys_newstat() (slot __NR_stat), andsys_stat64() (slot __NR_stat64), with the last being the most
          current.  A similar story applies for lstat(2) and fstat(2).

       *  Similarly, the defines __NR_oldolduname, __NR_olduname, and__NR_uname refer to the routines sys_olduname(), sys_uname() andsys_newuname().

       *  In Linux 2.0, a new version of vm86(2) appeared, with the old and
          the new kernel routines being named sys_vm86old() and sys_vm86().

       *  In Linux 2.4, a new version of getrlimit(2) appeared, with the old
          and the new kernel routines being named sys_old_getrlimit() (slot__NR_getrlimit) and sys_getrlimit() (slot __NR_ugetrlimit).

       *  Linux 2.4 increased the size of user and group IDs from 16 to 32
          bits.  To support this change, a range of system calls were added
          (e.g., chown32(2), getuid32(2), getgroups32(2), setresuid32(2)),
          superseding earlier calls of the same name without the "32"
          suffix.

       *  Linux 2.4 added support for applications on 32-bit architectures
          to access large files (i.e., files for which the sizes and file
          offsets can't be represented in 32 bits.)  To support this change,
          replacements were required for system calls that deal with file
          offsets and sizes.  Thus the following system calls were added:
          fcntl64(2), getdents64(2), stat64(2), statfs64(2), truncate64(2),
          and their analogs that work with file descriptors or symbolic
          links.  These system calls supersede the older system calls which,
          except in the case of the "stat" calls, have the same name without
          the "64" suffix.

          On newer platforms that only have 64-bit file access and 32-bit
          UIDs/GIDs (e.g., alpha, ia64, s390x, x86-64), there is just a
          single version of the UID/GID and file access system calls.  On
          platforms (typically, 32-bit platforms) where the *64 and *32
          calls exist, the other versions are obsolete.

       *  The rt_sig* calls were added in kernel 2.2 to support the addition
          of real-time signals (see signal(7)).  These system calls
          supersede the older system calls of the same name without the
          "rt_" prefix.

       *  The select(2) and mmap(2) system calls use five or more arguments,
          which caused problems in the way argument passing on the i386 used
          to be set up.  Thus, while other architectures have sys_select()
          and sys_mmap() corresponding to __NR_select and __NR_mmap, on i386
          one finds old_select() and old_mmap() (routines that use a pointer
          to an argument block) instead.  These days passing five arguments
          is not a problem any more, and there is a __NR__newselect that
          corresponds directly to sys_select() and similarly __NR_mmap2.

SEE ALSO         top

intro(2), syscall(2), unimplemented(2), errno(3), libc(7), vdso(7)

COLOPHON         top

       This page is part of release 4.11 of the Linux man-pages project.  A
       description of the project, information about reporting bugs, and the
       latest version of this page, can be found athttps://www.kernel.org/doc/man-pages/.

Pages that refer to this page: intro(2),  syscall(2),  unimplemented(2),  stapprobes(3stap),  libc(7),  man-pages(7),  vdso(7)

Copyright and license for this manual page

Welcome to Mossberg, a weekly commentary and reviews column on The Verge and Recode by veteran tech journalist Walt Mossberg, executive editor at The Verge and editor at large of Recode.

This is my last weekly column for The Verge and Recode— the last weekly column I plan to write anywhere. I’ve been doing these almost every week since 1991, starting at the Wall Street Journal, and during that time, I’ve been fortunate enough to get to know the makers of the tech revolution, and to ruminate — and sometimes to fulminate — about their creations.

Now, as I prepare to retire at the end of that very long and world-changing stretch, it seems appropriate to ponder the sweep of consumer technology in that period, and what we can expect next.

Let me start by revising the oft-quoted first line of my first Personal Technology column in the Journal on October 17, 1991: “Personal computers are just too hard to use, and it isn’t your fault.” It was true then, and for many, many years thereafter. Not only were the interfaces confusing, but most tech products demanded frequent tweaking and fixing of a type that required more technical skill than most people had, or cared to acquire. The whole field was new, and engineers weren’t designing products for normal people who had other talents and interests.

Dow Jones

But, over time, the products have gotten more reliable and easier to use, and the users more sophisticated. You can now hand an iPad to a 6-year-old and, with just a bit of help, she will very likely learn how to operate it quickly. That’s amazing, given that the iPad is far more powerful than any complex PC I was testing in the 1990s. Plus, today’s hardware and software rarely fails catastrophically like PCs did so often in the old days.

So, now, I’d say: “Personal technology is usually pretty easy to use, and, if it’s not, it’s not your fault.” The devices we’ve come to rely on, like PCs and phones, aren’t new anymore. They’re refined, built with regular users in mind, and they get better each year.

Anything really new is still too close to the engineers to be simple or reliable. Many people aren’t going to be able to hook up a dedicated virtual reality system, or want to wear the headset. And most of us can’t yet trust Siri, Alexa or Google Assistant for an accurate, useful answer much of the time. But it’s early days for those technologies.

So: Where are we now, and what’s coming?

The lull

As I write this, the personal tech world is bursting with possibility, but few new blockbuster, game-changing products are hitting the mainstream. So a strange kind of lull has set in.

The multi-touch smartphone, launched 10 years ago with Apple’s first iPhone, has conquered the world, and it’s not done getting better. It has, in fact, become the new personal computer. But it’s a maturing product that I doubt has huge improvements ahead of it. Tablets rose like a rocket but have struggled to find an essential place in many people’s lives. Desktops and laptops have become table stakes, part of the furniture.

The big software revolutions, like cloud computing, search engines and social networks, are also still growing and improving, but have become largely established.

Consumer drones and robotics are in their infancy, a niche, with too few practical uses as yet.

The biggest hardware and software arrival since the iPad in 2010 has been Amazon’s Echo voice-controlled intelligent speaker, powered by its Alexa software assistant. It arrived in 2015, and was followed last year by the similar Google Home device. I expect others.

Justin Sullivan / Getty

But the Echo and Alexa are just getting started. Amazon CEO Jeff Bezos told me in an interview last year that artificial intelligence was not just in the first inning of a long baseball game, but at the stage where the very first batter comes up. And, while Amazon doesn’t release sales figures for the Echo family, third-party estimates say that, while they are growing fast, they were still well below 10 million units last year. For comparison, even in a relatively weak period, Apple sold 50 million much costlier iPhones in just 90 days last quarter, and the combined total sales of the far more prevalent Android phones no doubt were much greater.

Google just announced that there are now two billion Android devices in active monthly use globally, and Apple announced a year and a half ago that there were over one billion iOS devices in active use. These are mostly smartphones, and they are no longer novel.

Wait for it

But just because you’re not seeing amazing new consumer tech products on Amazon, in the app stores, or at the Apple Store or Best Buy, that doesn’t mean the tech revolution is stuck or stopped. In fact, it’s just pausing to conquer some major new territory. And, if it succeeds, the results could be as big or bigger than the first consumer PCs were in the 1970s, or even the web in the 1990s and smartphones in the first decade of this century.

All of the major tech players, companies from other industries and startups whose names we don’t know yet are working away on some or all of the new major building blocks of the future. They are: Artificial intelligence / machine learning, augmented reality, virtual reality, robotics and drones, smart homes, self-driving cars, and digital health / wearables.

All of these things have dependencies in common. They include greater and more distributed computing power, new sensors, better networks, smarter voice and visual recognition, and software that’s simultaneously more intelligent and more secure.

Shutterstock

Examples of all these technologies already exist, but they are early, limited and mainly attractive to enthusiasts. Compared to what’s coming, they are like the Commodore PET (look it up, kids) or those huge car phones in old movies.

Ambient Computing

I expect that one end result of all this work will be that the technology, the computer inside all these things, will fade into the background. In some cases, it may entirely disappear, waiting to be activated by a voice command, a person entering the room, a change in blood chemistry, a shift in temperature, a motion. Maybe even just a thought.

Your whole home, office and car will be packed with these waiting computers and sensors. But they won’t be in your way, or perhaps even distinguishable as tech devices.

This is ambient computing, the transformation of the environment all around us with intelligence and capabilities that don’t seem to be there at all.

It reminds me of a great Saturday Night Liveskit from 2005, where cast member Fred Armisen, playing Steve Jobs, shows off an ever-smaller series of iPods, finally producing a model that’s literally invisible, yet holds “every photograph ever taken.”

Just in recent weeks, Facebook’s famed researcher Regina Dugan has announced that her secretive team is working on using the brain to type, and to control augmented reality devices. They are also developing ways to “hear” through your skin.

Their idea is that, even if augmented reality gets built into standard eyeglasses and can impose sophisticated virtual objects onto real life, it won’t be seamless if you have to push buttons, use touch controls or utter commands.

Apple reportedly has a secret project to monitor the glucose levels of diabetics with new noninvasive sensors, ending the need for daily test needles.

Google has changed its entire corporate mission to be “AI first” and, with Google Home and Google Assistant, to perform tasks via voice commands and eventually hold real, unstructured conversations.

Several small firms are pursuing the prospect of recharging mobile devices with power sent through the air, so power cords won’t be around.

I expect to see much of this new ambient computing world appear within 10 years, and all of it appear within 20.

Why it matters

Every one of these efforts has the potential to create a new world that’s unrecognizable. It’s a radically different way of thinking about tech.

When the internet first arrived, it was a discrete activity you performed on a discrete hunk of metal and plastic called a PC, using a discrete software program called a browser. Even now, though the net is like the electrical grid, powering many things, you still use a discrete device — a smartphone, say — to access it. Sure, you can summon some internet smarts through an Echo, but there’s still a device there, and you still have to know the magic words to say. We are a long way from the invisible, omnipresent computer in the Starship Enterprise.

Worse, those early computers were in your way. They were hulking objects that demanded space and skill. Even now, if you look around a restaurant, you’ll see smartphones on the tables, waiting to be used.

Computers have gotten vastly easier to use, but they still demand attention and care, from charging batteries to knowing which apps to use and when to use them.

Technology has been a great thing, but it’s been too unnatural, an add-on to life, for 40 years. What’s going on in the labs has the promise to change that.

The dark side

Some of you who’ve gotten this far are already recoiling at the idea of ambient computing. You’re focused on the prospects for invasion of privacy, for monetizing even more of your life, for government snooping and for even worse hacking than exists today. If the FBI can threaten a huge company like Apple over an iPhone passcode, what are your odds of protecting your future tech-dependent environment from government intrusion? If British hospitals have to shut down due to a ransomware attack, can online crooks lock you out of your house, office or car?

Good questions.

My best answer is that, if we are really going to turn over our homes, our cars, our health and more to private tech companies, on a scale never imagined, we need much, much stronger standards for security and privacy than now exist. Especially in the U.S., it’s time to stop dancing around the privacy and security issues and pass real, binding laws.

And, if ambient technology is to become as integrated into our lives as previous technological revolutions like wood joists, steel beams and engine blocks, we need to subject it to the digital equivalent of enforceable building codes and auto safety standards. Nothing less will do. And health? The current medical device standards will have to be even tougher, while still allowing for innovation.

The tech industry, which has long styled itself as a disruptor, will need to work hand in hand with government to craft these policies. And that might be a bigger challenge than developing the technology in the first place.

The oligopoly

Most of the work on this, especially what we and others can learn about and report about, is coming from the giant companies that make up today’s tech oligopoly— Apple, Amazon, Facebook, Google and Microsoft.

But, as tectonic shifts like this occur in technology, oligopolies get shaken up. For instance: today, Apple is the biggest of the group. By all reports, it’s working seriously on AR, self-driving cars and health initiatives. But its strict and admirable privacy policies make it harder for it to gather the vast amounts of data required for the best machine learning.

Microsoft is still trying to find a way to meld its formidable software and cloud expertise with a significant hardware business. The ad-based business models of Facebook and Google, now so dominant, could prove fickle. And Amazon has only had one really giant hardware hit — the Kindle — in its existence.

But, even if these oligarchs all do fine, and their ranks swell by one or two, the country and the world will have to ask if they have too much power — and, if so, how to curb it without killing progress.

The bottom line

We’ve all had a hell of a ride for the last few decades, no matter when you got on the roller coaster. It’s been exciting, enriching, transformative. But it’s also been about objects and processes. Soon, after a brief slowdown, the roller coaster will be accelerating faster than ever, only this time it’ll be about actual experiences, with much less emphasis on the way those experiences get made.

As a gadget-lover, this makes me a little sad. But, as a tech believer, it’s tremendously exciting. I won’t be reviewing all the new stuff anymore, but you can bet I’ll be closely watching this next turn of the wheel.

Thanks for reading. Mossberg out.

Quite a lot happening, and all of it bad on both sides.

Firstly it sounds like you were AWOL. In most jurisdictions your company could initiate (at a minimum) a formal disciplinary procedure against you for that. In some (and you have not said where you are) they could fire you with a letter.

but I only communicated to the CEO over the phone about the whole situation and I didn’t know how long I would be away. He was understanding and basically told me to take as much time as I needed.

That's unprofessional behavior by both of you. However the fact that your communicated by phone should be recorded somewhere, even if the words spoken are lost. It's not much, but it's something. Personally I would simply never let something like this rest on verbal communication - and I experienced a similar bereavement, BTW (condolences to you).

So you should assume that you're gone, one way or another, in these circumstances and proceed on that basis (i.e. get another job and assume this one is gone for good).

Now regarding the alleged forgery ...

As described it's a crime, possibly more than one, in most places.

It's possibly an identity theft. Someone impersonated you.

It's possibly an computer hacking crime (you suggest it's possible at least).

Note that :

I know there is a backdoor way to send emails using someone else’s account via some sort of a SQL database thing. We used to do it as jokes

That remark could make you complicit in hacking, joke or otherwise. Frankly I'd be surprised if it was not grounds for immediate dismissal anywhere - I'd certainly want you fired if I was working there. It hugely undermines your own ability to show fraud, as the fraud could just as easily be by you to cause trouble.

However, if you honestly believe this to be the case, it probably ought to be reported to the appropriate authorities.

As your own civil law position is dubious you should probably speak to a lawyer about this and ask their opinion on approaching the authorities.

On the civil law side, it's simply beyond my ability to even guess what a court would say, but there's a legal principle called the doctrine of clean hands which may apply. Your company may have accepted (accepting your claim as true) a fake email as valid, but it sounds like you were AWOl as well and there's the hacking-as-normal-behavior issue. A court could consider that both of you were breaking the rules and just kick you both out.

If you want to sort this out you will need a lawyer (and one who knows contract law and employment law, not just someone who handles house sales).

Personally I would be loath to let someone impersonate me and force my resignation under any circumstances. It's a crime, and it ought to be investigated and, ideally, someone convicted for it (again taking you at face value). However you've put yourself in a very difficult position and this won't, I suspect, be a clean fight.

There are days in this profession in which I am surprised. The longer I stay in the technology industry, they become further and further apart. There are several reasons to be surprised: someone comes out of the blue with a revolutionary product and the ecosystem/infrastructure to back it up, or a company goes above and beyond a recent mediocre pace to take on the incumbents (with or without significant financial backing). One reason is confusion, as to why such a product would ever be thought of, and another is seeing how one company reacts to another.

We’ve been expecting the next high-end desktop version of Skylake for almost 18 months now, and fully expected it to be an iterative update over Broadwell-E: a couple more cores, a few more dollars, a new socket, and done. Intel has surprised us with at least two of the reasons above: Skylake-X will increase the core count of Intel’s HEDT platform from 10 to 18.

The Skylake-X announcement is a lot to unpack, and there are several elements to the equation. Let’s start with familiar territory: the first half of the processor launch.

The last generation, Broadwell-E, offered four processors: two six-core parts, an eight-core part, and a top-tier 10-core processor. The main difference between the two six-core parts was the PCIe lane count, and aside from the hike in pricing for the top-end SKU, these were iterative updates over Haswell-E: two more cores for the top processor.

This strategy from Intel is derived from what they call internally as their ‘LCC’ core, standing for ‘low core count’. The enterprise line from Intel has three designs for their silicon – a low core count, a high core count, and an extreme core count: LCC, HCC, and XCC respectively. All the processors in the enterprise line are typically made from these three silicon maps: a 10-core LCC silicon die, for example, can have two cores disabled to be an 8-core. Or a 22-core XCC die can have all but four cores disabled, but still retain access to all the L3 cache, to have an XCC processor that has a massive cache structure. For the consumer HEDT platform, such as Haswell-E and Broadwell-E, the processors made public were all derived from the LCC silicon.

The first half of the Skylake-X processor llineup follows this trend. Intel will launch four Skylake-X processors based on the LCC die, which for this platform will have a maximum of 12 cores. All processors will have hyperthreading.

The bottom processor is the Core i7-7800X, running at 3.5 GHz with a 4.0 GHz turbo. This design will not feature Intel’s new ‘favored core’ Turbo 3.0 technology (more on that below), but will have six cores, support quad-channel memory at DDR4-2400, come in at a TDP of 140W, have 28 PCIe lanes, and retail for around $400. This processor will be the entry level model, for any user who needs the benefit of quad-channel memory but perhaps doesn’t need a two-digit number of cores or has a more limited budget.

Next up is the Core i7-7820X, which hits a potential sweet spot in the LCC design. This is an eight-core processor, with the highest LCC base clock of 3.6 GHz and the joint-highest turbo settings: 4.3 GHz for regular turbo and 4.5 GHz for favored core. Unlike the previous processor, this CPU gets support for DDR4-2666 memory.

However in another break from Intel’s regular strategy, this CPU will only support 28 PCIe lanes. Normally only the lowest CPU of the HEDT stack would be adjusted in this way, but Intel is using the PCIe lane allocation as another differentiator as a user considers which processor in the stack to go for. This CPU also runs in at 140W, and comes in at $600. At this price, we would expect it to be competing directly against AMD’s Ryzen 7 1800X, which will be the equivalent of a generation behind in IPC but $100 cheaper.

The third processor is also a change for Intel. Here is the first processor bearing the new Core i9 family. Previously we had Core i3, i5 and i7 for several generations. This time out, Intel deems it necessary to add another layer of differentiation in the naming, so the Core i9 naming scheme was the obvious choice. If we look at what the Core i9 name brings to the table, the obvious improvement is PCIe lanes: Core i7 processors will have 28 PCIe lanes, while Core i9 processors will have 44 PCIe lanes. This makes configuring an X299 motherboard a little difficult: see our piece on X299 to read up on why.

Right now the Core i9-7900X is the only Core i9 with any details: this is a ten core processor, running with a 3.3 GHz base, a 4.3 GHz turbo and a 4.5 GHz favored core. Like the last processor, it will support DDR4-2666 and has a TDP of 140W. At this level, Intel is now going to charge $100/core, so this 10-core part runs in at a $999 tray price ($1049 retail likely).

One brain cell to twitch when reading this specification is the price. For Ivy Bridge-E, the top SKU was $999 for six-cores. For Haswell-E, the top SKU was $999 for eight-cores. For Broadwell-E, we expected the top SKU for 10-cores to be $999, but Intel pushed the price up to $1721, due to the way the enterprise processors were priced. For Skylake-X, the new pricing scheme is somewhat scrapped again. This 10-core part is now $999, which is what we expected the Broadwell-E based Core i7-6950X to be. This isn’t the top SKU, but the pricing comes back down to reasonable levels.

Meanwhile for the initial launch of Skylake-X, it is worth noting that this 10-core CPU, the Core i9-7900X, will be the first one available to purch. More on that later.

Still covering the LCC core designs, the final processor in this stack is the Core i9-7920X. This processor will be coming out later in the year, likely during the summer, but it will be a 12-core processor on the same LGA2066 socket for $1199 (retail ~$1279), being part of the $100/core mantra. We are told that Intel is still validating the frequencies of this CPU to find a good balance of performance and power, although we understand that it might be 165W rather than 140W, as Intel’s pre-briefing explained that the whole X299 motherboard set should be ready to support 165W processors.

In the enterprise space, or at least in previous generations, Intel has always had that processor that consumed more power than the rest. This was usually called the ‘workstation’ processor, designed to be in a single or dual socket design but with a pumped up frequency and price to match. In order for Intel to provide this 12-core processor to customers, as the top end of the LCC silicon, it has to be performant, power efficient, and come in at reasonable yields. There’s a chance that not all the factors are in place yet, especially if they come out with a 12-core part that is clocked high and could potentially absorb some of their enterprise sales.

Given the expected timing and launch for this processor, as mentioned we were expecting mid-summer, that would have normally put the crosshairs into Intel’s annual IDF conference in mid-August, although that conference has now been canned. There are a few gaming events around that time to which Intel may decide to align the launch to.

Part One: Five Billion Shillings

Today, someone — probably in India or Africa — is purchasing their first mobile. It will likely be a prepaid account, and possibly will be a device that has multiple SIMs, because it’s often much cheaper to make a call on your own carrier than across to another carrier.

That phone will likely look a lot like the Nokia 3310 — what we used to call a mobile, but which we now refer to as a ‘dumbphone’.

It’s getting hard to remember, but a decade ago — before iPhone — all our mobiles were dumbphones. We’d use them to call or text and perhaps play the occasional game of snake. But that’s it. That’s all we could do.

Yet even that completely changed our lives. And that’s just as true in Africa as in Australia. Actually, it’s more true in Africa.

In Kenya, back in 2007, the dominant carrier — Safaricom — launched a service for the millions of mobile-owning of Kenyans.

M-PESA allows you to send money via a text message.

How does that work?

Before I explain that, you have to understand something about East Africa, and about the developing world generally — there aren’t a lot of banks, there aren’t a lot of bank branches, and most people are ‘unbanked’ — they don’t have a bank account.

Why is this? In the developing world, banks really only want to deal with rich companies and rich individuals. Even the average SME in Kenya doesn’t have enough money to make it worthwhile for a bank to do business with them.

All that meant that Kenya had a huge cash economy in 2007 — simply because there was no alternative.

M-PESA became that alternative.

Safaricom already had a network of agents throughout Kenya, who would sell you a phone or a SIM or calling minutes. These agents were authorised to receive cash as well, that could then be deposited with an individual’s M-PESA ‘account’ — a savings account connected to their SIM.

Once that cash had been deposited with an agent, it can be transferred via a text message, to anyone else with M-PESA. Once transferred, the recipient could visit an agent and receive the cash.

That might not sound like much, but in Kenya — where people migrate from villages to Nairobi for work — it means the world.

Instead of spending a full day every fortnight traveling back to that village to hand cash over to a family member, the cash could be transferred instantly — and much more safely.

M-PESA became an immediate, roaring success. Peer-to-peer bank transfers — something we rarely use here in Australia transformed the Kenyan economy, because it became so much easier for people to move their money. That added velocity to money, and — over the last decade — has given Kenya one of the fastest growth rates in the world.

Although ideal for person-to-person money transfers, M-PESA hadn’t been designed for commercial transactions.

You couldn’t walk into a corner store and pay for a packet of laundry detergent using M-PESA. You’d need to visit an agent, get some cash, then go to the corner store, cash in hand. And if if you were that shop owner, you had to pay your wholesalers in cash — which means you had to have a lot of cash on hand — making every SME was an attractive target for robbers.

One startup thought they had a solution for that shortcoming.

Kopo Kopo created a merchant services interface to M-PESA, allowing SMEs from the corner store to electrician and every other kind to have business M-PESA accounts.

When Kopo Kopo launched, Kenya began a slow transition to a cashless society — and they did this without banks, without credit cards, without tap-and-go payments. All anyone needs is a dumbphone, because Kopo Kopo realised they could build a monster of a business atop the APIs offered by M-PESA.

That was just the prelude. As Kopo Kopo signed up merchants, and those merchants began to receive payments via M-PESA, they created something they’d never had before — a verifiable digital ledger of transactions. Those transactions might not cover all of the cash flow of the business — Kenyans continue to use banknotes — but it captured enough of a snapshot that the ledger itself became a useful source of data.

Kopo Kopo realised they’d unlocked a fantastically rich source of analytics for these businesses. Immediately Kopo Kopo offered that data to the businesses, so they could understand their cash flow, their cash needs, and their cash trajectory.

Keep in mind that all of these SMEs were (and mostly still are) unbanked. That makes an awareness of their cash trajectory is even more important, because they couldn’t run to the bank for a bit of help through a rough patch.

When Kopo Kopo realised this, they took the next logical step — prequalifing SMEs for lines of credit.

Let’s unroll this: these SMEs are unbanked because it’s too expensive for a bank to do the lengthy credit checks required to ensure the business is an appropriate capital risk. The costs of doing that are far higher than any profit realised from making a loan to the business.

But Kopo Kopo had all the data it needed in hand to be able to assess the creditworthiness of the business.

It could use its analytics — based on that business’ stream of transactions — to understand whether the business qualified for a line of credit. That took all of the costs away from the bank, and with Kopo Kopo as market-maker, connecting SME to bank, credit began to flow into Kenyan SMEs that had never had access to working capital.

In 2017, Kenyans have a banking system that is in some significant ways more sophisticated than anything we have here in Australia. And they did it all themselves.

Part Two: Three Act Play

Everything that’s happened in Kenya is happening in Australia.

It’s not exactly the same, of course — by and large Australians and Australian businesses have bank accounts, access to capital, and a variety of payment systems — from cheques to EFTPOS to BPAY to tap-and-pay. So some of the drivers in Kenya simply aren’t present here.

But we have other drivers.

At the ten-year anniversary of the introduction of the iPhone, well north of eighty percent of adult Australians carry a smartphone. What started as simply a clever device for combined music, messages and voice calls into a single device has become the universal tool.

And I do mean universal: By 2020 eighty percent of all adults everywhere in the world will use a smartphone. Kenya will catch up to Australia.

In Australia, the world of commerce remains curiously disconnected from the smartphone. You probably have a banking app, but how many of your transactions actually pass through your smartphone?

Even now — in 2017 — it’s not uncommon to encounter a web page that hasn’t been redesigned to work well on a smartphone. It’s as though these devices have become so common they get ignored.

But the truth is a bit more brutal — we haven’t yet climbed that first hill. Many of Australia’s businesses haven’t even made it to the lowest rung of the ladder — simple APIs that make the Web accessible on mobiles.

Some companies have gone right around the Web, creating their own apps to capture commerce. But that introduces its own problems, because there are big roadblocks to commerce on the smartphone.

If an Australian company writes an app that allows a purchase to be made — and paid for, via the app — it’s more likely than not that Apple or Google will appear, asking for a big slice of that sale.

Companies can get around this by holding your credit card details — keeping the transaction away from the smartphone. But that’s an obstacle in itself, because you have to get the customer to enter those details within the app — and the customer has to trust the seller before they’re willing to do that.

All of this is fine if you’re buying an album from iTunes, but what if you’re buying a sofa from IKEA? Payments platforms like PayPal try to bridge this gap — keep your details in PayPal, then use your PayPal account to make a payment to any retailer. That works if you have a PayPal account — and the merchant accepts PayPal. But if the edges don’t meet — and that’s often the case — customer and merchant are frustrated.

When you consider that by the early 2020s the majority of all online transactions will happen through a smartphone, these payments failures constitute a looming catastrophe for Australian commerce.

Fortunately there’s a solution.

Eighteen months ago the World Wide Web Consortium (W3C) launched a Web Payments Working Group, chartered to bring a rich set of APIs to the browser — and in particular, the mobile browser — so the user enters their payment details once, in the browser, then can present those payment details to any compatible mobile website.

With Web Payments, a website has more capability than an app, and creates a payment mechanism that doesn’t route the payment through Apple or Google. Intriguingly, the payment mechanism is now being extended so it can be any payments mechanism both buyer and seller agree to — it could be a credit card, or PayPal or AliPay or bitcoins, or… whatever you can dream up.

That doesn’t mean you’ll be able to take your Ethereum and use them to pay for groceries at Coles. But it does mean that you can easily make payments in whatever currency you choose to use, with any payments processor you choose to use, so long as the seller agrees.

Right away you can see that the Web Payments API is going to create an explosive growth in ‘exchanges’ that will take a currency or payment mechanism and convert it into something compatible with a particular seller.

Just as Kopo Kopo recognized merchant services as the missing element in M-PESA, smart companies will be seizing the moment with Web Payments, building these marketplaces, and enabling very broad commercial connections between buyers and sellers of all different stripes.

And of course you folks, as experts in APIs, will be going to all of your clients and offering to integrate Web Payments into their mobile websites and their mobile apps — or will, as soon as it’s in the mobile browser. Right now only Chrome Android v53 supports this API, and while it will clearly spread to other mobile browsers — they’ve all supported this standardisation effort — it may be 18 to 24 months before it becomes relatively ubiquitous.

That gives us a great moment to think about how to rethink payments from the customer’s point of view. Payments have been constrained by payment mechanism — credit cards work one way, PayPal works another, BPAY is weirder still. Each of them had their own user experience. All of that can be hidden away now, making it all seamless. That opens new possibilities for all sorts of payments systems.

Consider micropayments. From the birth of the Web, micropayments have been held out as a holy grail, the salvation of media. Instead of the New York Times or the Guardian fighting with you for a yearly subscription, they’d charge you a few cents for each article.

That’s never been possible because the transaction processing fees for payments have always been far greater than any amount any reader would be willing to pay to read a single article.

There have been a lot of attempts to solve the micropayments problem — and all of them have required some complication to the user experience. No one wants that, so micropayments have never taken off.

With the Web Payments API, a micropayments system can be just another payments interface. Hit a news site, it requests a payment using a micropayments service via the API, and you’re on your way. I don’t know if that can be entirely frictionless — you clearly want a user to authorize a transaction, even if it’s only a few cents — but it can be nearly frictionless.

Of course, someone will have to setup a micropayments processor — and that’s something that perhaps the publishers could do as a non-profit financial institution whose sole purpose is to hold the funds used to make micropayments. There’s precedent for this: Safaricom is required by Kenya’s central bank to hold all M-PESA funds in a special bank account that can’t be used for any other purpose.

This is the most interesting thing about payments APIs. As soon as you set up a new payments mechanism, an ecosystem of related applications constellate around it.

A Web Payments API means micropayment. It means micropayments processors. It means micropayments accepting websites. It means micropayments analytics tools. And on and on and on.

The Web Payments API is not a culmination of commerce on the Web. On the contrary it’s just the beginning. Once you have a mobile API, you can start to build services around it, and services on top of it. That’s the lesson of M-PESA, and we’ll see it happen all over again — at global scale — with Web Payments.

Things are about to get a whole lot more interesting.

Part Three: Without Peer

As an advanced economy Australia already has all sorts of whizzy payment systems — we’re even getting Apple Pay, someday — but remarkably all of these systems sit atop a foundation that is over thirty years old. Back in the mid-1980s the Reserve Bank led an effort to move the nation’s payment infrastructure to an electronic network.

That payment system keeps bankers hours — only clearing payments during the working week — and isn’t anything like instantaneous. Unless you’re moving funds within a bank, it takes hours or days to make a payment using any of Australia’s payment systems. Sure, your bank will cover you when you use your tap-and-pay card — that’s what you’re paying for, in part — but those payments won’t be processed, credited and debited until the payments as a whole are processed in a ‘batch’, generally overnight.

We rarely hear the term ‘batch processing’ anymore, but until the late 1980s that was the way most computer systems were designed. They ran one program at a time, ran through all the data, then moved on to the next program. So of course our payments processing system worked in batches.

Then along came the modern PC, which can do many dozens of things simultaneously. And the Internet, which sent messages to-and-fro 24 hours a day.

And, of course, the smartphone, which is the mash-up of the PC and the internet — constantly connected and constantly doing many different things.

This goes some way to explain why our 35 year old national payments system isn’t just looking a little long in the tooth, it’s actually hindering innovation. Australians have tens of millions of smartphones that integrate poorly with the payments system.

We’re not alone in this. Even the most advanced national payments systems — such as Singapore — remain separate from all those smartphones. Although the smartphone has become the most important piece of technology most people own, that hasn’t been reflected in payments.

Yet.

In October, the New Payments Platform (NPP), a multi-year initiative of the Reserve Bank, will be released to the public. Unlike any previous Australia payments platform, the NPP has been designed from the ground up as an open platform. It’s a set of APIs that any company can use to build their own payments services.

Right now only a very few firms — the big banks, and a few other payments processors — have access to Australia’s payments infrastructure. And, has already been mentioned, it’s slow. The NPP promises settlement within minutes — and mostly within a few seconds. So it’s not only open, it’s fast.

What can we do with that?

The first thing you’ll see is a whole crop of peer-to-peer payment applications. That’s right, the first thing we’ll do with NPP is replicate the functionality of Kenya’s M-PESA.

Why exactly Australians need a peer-to-peer payments system isn’t exactly clear, but we’ll have several to choose from.

However, there’s another type of peer-to-peer payments that looks very interesting indeed: payments passing between smartphone apps.

Right now, your smartphone apps don’t really talk to one another. And they certainly don’t trade with one another.

Let’s say a restaurant booking app wanted to get your a taxi to take you door to door — right now that would all happen behind the scenes, because that booking app can’t order and pay for that taxi. There’s no API allowing these apps to perform transactions.

In every case these apps either have to go through Apple or Google — with their 30% cut — or they have to rely on credit card information that the user may not want to enter. And these apps only do that for themselves. One app can’t pay another.

But if a peer-to-peer payment can move cash between two individuals, why couldn’t it also be used to move cash between two apps owned by the same individual?

All those apps need is an API to a payments service that allows these apps to send payments to one another instantly. With that API apps will be free to perform their own commercial operations.

Why would we want to add that capacity to an app? I’ve already given one example, where you get a restaurant and taxi booking together.

But once you have the capacity to perform these intra-smartphone transactions, you’ll see a new class of apps that provide little pieces of a value chain — a value chain connected via peer-to-peer payments.

This is not how we think of smartphones today. Apps are little islands, each of them having very little to do with any other. But payments are the golden carrot, the lure that provides all the reason any developer or merchant needs to open that capacity up to other apps. Today there’s no reason, but in October there will be an overwhelming reason: dollars.

You have seven months until the NPP is released, and all of the documentation you need to use to write services that make these APIs available to apps has already been released. It’s my hope that at least one person in the room today gets the opportunity that’s opening up here, to create a new economy around smartphones.

And it’s not just opening up here.

The NPP is being built by SWIFT. You’ve heard of them before — because when you send money overseas, you use a SWIFT code to transfer the funds. They’re the international payments network for the banks. They’re building the NPP here in Australia, and once they’ve got it all worked out and debugged, they’ll be offering it to every other country that wants a 21st century payments platform.

Creating a peer-to-peer payments platform for smartphone apps using the NPP isn’t just a neat solution for Australians — it’s a bridgehead to a global solution for intra-smartphone payments.

By 2020, more than five billion adults will be using smartphones. As these folks get their own versions of the NPP, a smartphone payments platform will travel with them. It will become the foundation for commerce in the middle of the 21st century.

That’s absolutely going to happen. The question before you is simple — do you want to be the person to make that happen? You have the insight, you have the tools, and you have the APIs.

What else do you need?

Wesley Reisz talks to Sachin Kulkarni, Director of Engineering at Facebook, about the engineering challenges of building Facebook Live, and how it compares to the video upload platform at Facebook.

Facebook Infrastructure

• 1:48 - Facebook infra. powers the board family of apps including the Facebook app, Messenger and Instagram. The group is responsible for storage, caching, pub/sub, monitoring, streaming and so on.
• 2:30 - The video infra team builds the video infrastructure across the whole company. Projects include a distributed video encoding platform which results in low latency video encoding, video upload and ingest. Ingesting is about moving the bytes from the client apps to the Facebook data centre, while encoding is about server side processing to reduce size while keeping the quality high.
• 2:58 - Another angle is video clustering, where similar videos can be clustered together for better search ranking.

Facebook Live encoding

• 3:35 - Facebook Live does encoding on both the client and the server.
• 4:03 - The trade-off between encoding on the client-side and the server-side is mostly around video quality vs. latency and reliability. Since encoding is typically lossy if the network is good the Facebook app will try to keep the quality as high as possible and use little or no encoding. Conversely if the network is less good then more encoding is done on the phone to keep the amount of data to be transferred smaller.

Video at Facebook Scale

• 4:55 - There are 1.28 billion daily users of Facebook. A good example of where this causes problems is with comparably rare situations such as race conditions because the normally rare cases will get hit more frequently with that volume of users. In consequence avoiding race conditions needs to be thought of at design time.
• 6:15 - Facebook launches everything on an experimental basis starting with internal users. Then roll-out to the wider public is gradual - 0.1% of users, then 1%, 5% and so on to expose race conditions and other issues early.
• 6:56 - For back-end systems the release is typically done weekly so the release goes from just a handful of users to 1.28 billion users in a week.

Facebook Live

• 8:28 - Facebook Live is a live streaming platform open to all users. The latency depends a lot on where the broadcaster and viewer are, and the network conditions in both of those places. The aim though is to keep the latency in single digit milliseconds.
• 9:44 - The product started at a hackathon in 2015. A small team built a working prototype infrastructure in just a few days. The first thing they streamed was a clock to measure the latency of the system!
• 11:30 - It took around 4 months to get from the prototype to launching Live to public profiles in August of 2015. By December 2015 the platform was scaled to all users on iOS, Android and browsers.
• 12:04 - It was possible to build the product that quickly because a lot of infrastructure already existed - the Everstore BLOB storage system is solid, and they could also rely on open-source software like NGINX for doing the encoding and processing.
• 13:14 - Facebook Infrastructure is largely a C++ shop. There is some Java and Python, and the business logic is all done in PHP. The iOS apps are written in Objective C and the Android apps are in Java.

Facebook Live Architecture

• 13:54 - It all starts with the broadcast client - this could be an Android or iOS app, the Mentions app, or via the Live API. In the client app there are libraries which do packaging, encoding and so on.
• 14:13 - The stream is sent via RTMPS (Real-Time Messaging Protocol) to a geographically local PoP. Then the connection is forwarded over an internal Facebook network to a Facebook data-centre.
• 14:34 - At the data-centre the stream hits an encoding host which authenticates the stream, encodes it into multiple formats at different bitrates, and packages it into RTMP or DASH (Dynamic Adaptive Streaming over HTTP). The stream is then cached in a CDN before it hits the player.
• 15:27 - Users can be broadcasting from anywhere in the world so the geographically local PoP reduces round-trip time.
• 15:44 - A key thing for load balancing is hashing based on the stream ID. When you make a request to go live you get a stream ID and a URI. Facebook does a hash based on the stream ID and maps the stream to different data centres based on the hash.
• 16:13 - The client libraries run a speed test on iOS and Android to figure out the video bitrate to be used for encoding. They then encode the uncompressed bitstreams from the phone using the H.264 and AAC codecs for video and audio before warping the compressed frames in an RTMP compatible format and sending the packets to the server.
• 17:03 - Network bandwidth is not a static thing and can change during the broadcast. Facebook Live uses Adaptive Bitrate to cope with this.
• 19:57 - To stream the live stream out they use MPEG-DASH, an adaptive bitrate streaming format that enables streaming data over HTTP. It comprises a manifest file, essentially an index which points to media files, and individual media files, for example for each second of the live stream.
• 20:43 - When you see a live stream in your feed and you click on it the player requests the manifest. If it isn't already on your local PoP the request goes to the data centre to get the manifest, and then fetches the media files. As they get sent back they are cached on the PoP if they aren’t there already.

Video Upload

• 24:10 - One of the key challenges with Live is that it has to happen in real-time. For video upload you can batch the workload and hence latency is less key.
• 25:08 - They key requirements for building a stable and scalable video encoding platform is that it needs to be fast, flexible, able to cope with spikes, and very efficient.
• 25:28 - At a high level the client library takes the video and breaks it up into smaller chunks corresponding to GOPs (Groups Of Pictures) roughly equivalent to a scene in a video which is sent to the server. On the server-side a pre-processor receives the chunks and writes them to a cache and then starts encoding them in parallel as the chunks arrive. Facebook gets around 10x speed-up by encoding in parallel compared to doing this serially.

AI Encoding

• 26:25 - Facebook also tries to be efficient using bitrates since this affects user’s data plans. Creating smaller files without using an exorbitant amount of CPU is a hard problem because modern encoders have so many combinations of encoding settings.
• 27:17 - The key insight is that not only is each video is different, but also that even within a video the encoding settings could be different for each scene.
• 27:53 - Facebook uses AI to optimise the encoding settings. A training set is used to train a neural net model which can then come up with the right settings for each scene. The disturbed encoding system naturally lends itself to this kind of approach.
• 28:27 - AI encoding resulted in 20% smaller files than H.264.

New Live Features

• 28:55 - Facebook Live now allows people watching the stream to be invited to join it directly and ask questions during the stream. This requires very low latency - in the order of 100s of Milliseconds.
• 29:49 - This is done using a different protocol - WebRTC - which is typically used for video calling.

Resources

More about our podcasts

You can keep up-to-date with the podcasts via our RSS feed, and they are available via SoundCloud and iTunes.  From this page you also have access to our recorded show notes.  They all have clickable links that will take you directly to that part of the audio.

Previous podcasts

May 31, 2017
John Boardley

Recipes are as old as eating and recorded recipes date back to the invention of writing, with the most ancient examples from Mesopotamia, written in Akkadian cuneiform and dating to about 1750 BC. From late Imperial Rome, a collection of recipes from the late fourth or early fifth century, commonly referred to as Apicius, has survived via eighth- and ninth-century manuscript copies. Moreover, dozens of recipe books have survived from the Middle Ages, including the tenth-century Baghdadi cookbook by Ibn Sayyar al-Warraq and the fourteeth-century treatise on cookery and cooking techniques, La Viandier, written by Guillaume Tirel, chef to the Court of Charles V of France. The book includes a section on dishes for the sick, in which barley gruel and, naturally, chicken soup is recommended.

One of the very earliest English-language cookery manuscripts, The Forme of Cury, dates to about 1390 and was authored by cooks to king Richard II. Among its 196 recipes is one for ‘blanc mang’, forebear of the jelly-like British dessert, blancmange. The Middle English word ‘cury’ is from the old French word for ‘cookery.’

Modern cooking has its origins in Renaissance Italy and, undoubtably, the most famous of fifteenth-century chefs was Martino de Como, or Maestro Martino. He worked for Cardinal Ludovico Trevisan (1401–65), a cleric famed for his fine food and entertaining. In a letter of 1450 the Cardinal mentions the very recent conquest of Milan at the hands of Francesco Sforza (a truly momentous event) as a mere footnote to talk of procuring the finest fish for an upcoming dinner held in honor of the visiting pope, Nicholas V. Maestro Martino, described by his peers as a prince of cooking, compiled Libro de Arte Coquinaria (‘The Art of Cooking’), a book that marks a watershed in the transition from medieval to Renaissance, or early modern, cuisine.

However, the very first printed cookbook was published by Laurentius de Aquila and Sibylinus Umber in Venice, in June 1475. At about the same time an edition was published in Rome in the typeface of Ulrich Han (and it has been suggested that the book was published by an anonymous printer with types discarded by Han). The text, written by Bartolomeo Sacchi, better known by his pseudonym, Platina, owes much to the recipes of Martino, whom he acknowledges in the introduction. Platina’s cookery book, De honestate voluptate et valitudine (‘On Right Pleasure & Good Health’) was an immediate success. It was issued in at least eight editions in Italy before 1500 and was even cited by Leonardo da Vinci.

The appearance of the first printed cookbooks in the fifteenth century coincided with increased use of the fork. Although the fork had been known prior to antiquity, its use was mostly restricted to the kitchen for carving and serving. One story of its introduction to the European table credits an eleventh-century Byzantine princess who came to Venice to marry. Her use of a fork at the table caused quite a stir, with one observer noting that such a thing was “luxurious beyond belief”; and Venetian Church leaders condemned such gross affectations, declaiming, “God in his wisdom has provided man with natural forks – his fingers. Therefore it is an insult to Him to substitute artificial metallic forks for them when eating.” (Giblin, p. 46) By the fifteenth century its use in Italy was common. It spread to France via Catherine de Médici who brought them to Paris in 1533, and was in common use throughout much of Europe by the late seventeenth century.

Not only were most of the earliest cookbooks rather vague on precise cooking techniques and measures, but they were not illustrated. Almost a hundred years would pass before the appearance of the first illustrated cookbook in 1570, with Bartolomeo Scappi’s Opera published by the Tramezzino press in Venice and supplied with 28 fine copperplate engravings (in some later editions substituted by woodcuts). Scappi was chef to cardinals and personal cook to two popes, Pius IV and Pius V. Before diving into the recipes, the author begins by outlining rather detailed instructions for setting up a kitchen. Typographically, the book is very well organized. The entire volume, running to some 900 pages, is set in a good italic type with italic all-caps for running heads, two-line roman drop-caps and a small number of ten-line decorated initials, plus the modest use of arabesque ornament – overall a fine example of typographic hierarchy achieved through a minimum of type styles and accoutrements.

Today, cookbooks are frequent visitors to best-seller lists and their celebrity-chef authors are household names. However, collections of recipes, whether from a 4000-year-old Babylonian clay tablet or from a contemporary full-color, illustrated, typographic book, are testaments to our unfailing and undiminishing passion for preparing and enjoying food.■