For 52 straight days this winter, Shannon Hovey woke up in the company of five other men in a metal tube, 20 feet long and seven feet in diameter, tucked deep inside a ship in the Gulf of Mexico. He retrieved his breakfast from a hatch (usually eggs), read a briefing for the day, and listened for a disembodied voice to tell him when it was time to put on a rubber suit and get to work. Life in the tube was built around going through these same steps day after day after day … while trying not to think about the fact that any unintended breach in his temporary metal home would mean a fast, agonizing death.

Hovey works in one of the least known, most dangerous, and, frankly, most bizarre professions on Earth. He is a saturation diver—one of the men (just about all have been men*) who do construction and demolition work at depths up to 1,000 feet or more below the surface of the ocean.

Diving to that depth—or just about any depth—involves breathing pressurized air. Inert gases in it, such as nitrogen, dissolve benignly into your blood and tissues—as long as the weight of all the water above you keeps them compressed. But when you want to return to the surface, that gas needs time to diffuse out slowly. If not, if a diver shot straight to the surface, the gas would form bubbles, like in a shaken can of soda. Inside that diver’s body, it would be as if millions of tiny explosives began to detonate. Known as the bends or, more technically, decompression sickness, the condition can be catastrophically painful and debilitating, and, depending on the depth, nearly impossible to survive. Diving to 250 feet for an hour, for example, would require a five-hour ascent to avoid getting even slightly bent. (The condition was first seen in the 19th century, when men leaving pressurized caissons, used to dig tunnels and build bridges, mysteriously took ill and began dying.)

The world—and, specifically, the oil and gas industry—needs commercial divers like Hovey who can go to the seabed to perform the delicate maneuvers required to put together, maintain, and disassemble offshore wells, rigs, and pipelines, everything from flipping flow valves, to tightening bolts with hydraulic jacks, to working in tight confines around a blowout preventer. Remotely operated vehicles don’t have the touch, maneuverability, or judgment for the job. And so, a solution. Experiments in the 1930s showed that, after a certain time at pressure, divers’ bodies become fully saturated with inert gas, and they can remain at that pressure indefinitely, provided they get one long decompression at the end. In 1964, naval aquanauts occupied the first Sea Lab—a metal-encased living quarters lowered to a depth of 192 feet. The aquanauts could move effortlessly between their pressurized underwater home and the surrounding water, and they demonstrated the enormous commercial potential of saturation diving. It soon became apparent that it would be easier and cheaper to monitor and support the divers if the pressurized living quarters weren’t themselves at the bottom of the sea. At this moment, all around the world, there are commercial divers living at pressure inside saturation systems (mostly on ships, occasionally on rigs or barges), and commuting to and from their jobsites in pressurized diving bells. They can each put in solid six-hour working days on the bottom.

Hovey and his fellow divers spent that six-week assignment working at the relatively shallow (but still quite deadly) depth of 250 feet, and living in a shipboard capsule pressurized to the same level. Pressure can be measured in atmospheres (atm) or pounds per square inch (psi). Pressure at sea level is 1 atm, or 14.7 psi. Inside a bicycle tire is about 65 psi. Hovey was living at over 110 psi. An ocean-and-a-half away, diver Steve Tweddle was making his way through a 28-day job in “storage,” as they call it, for work at a depth of 426 feet (190 psi) in the North Sea. The Gulf of Mexico and the North Sea share a history of offshore drilling, sparked by the worldwide oil crisis of the 1970s, which sent prices skyrocketing and saw offshore oil and gas rigs pop up like giant galvanized lily pads. The vast majority of saturation dives are for maintaining or taking down this oil and gas infrastructure.

A saturation diver starts a job when he leaves the “beach” (any solid ground) and steps onto a flat-bottomed ship known as a dive support vessel (DSV). Every piece of equipment and person on the ship is there to support the work and lives of the divers. There are subsea managers and dive supervisors, life support supervisors, life support technicians, and assistant life support technicians. They control what the divers breathe and eat, supply personal necessities, and even help remotely flush the toilet—whatever is necessary to keep them comfortable (such as it is) and alive.

Before taking up residence in the saturation chamber, the core living space of the system, every diver must pass a medical workup including, among other things, a hunt for any signs of infection. Even a simple cold can be incredibly hazardous to a saturation diver—clogged ears and sinuses trap air that the divers won’t be able to equalize to the pressurized air, potentially causing permanent damage that can end a career. Before entering the sat system, Tweddle always takes a shower with antibacterial soap to scrub off any hitchhiking germs.

In their last moments before a job, both Hovey and Tweddle call their families, even though they can use cell phones while in the chamber. Hovey, a 42-year-old American with intense, mournful eyes and a gray-dappled beard, used to work as a sound engineer and is now (most likely) the only saturation diver who works as an herbalist when on the beach. He tries to find a sunny spot on the ship’s helipad for his phone call. It’s his last chance to breathe fresh air, and his last chance to speak in a voice his family will understand. Once he’s at pressure, he’s going to sound like Donald Duck after huffing a roomful of helium balloons.

Air—compressed or otherwise—is about 21 percent oxygen, 78 percent nitrogen, and one percent everything else. Below about 100 feet divers breathing compressed air, including recreational scuba divers, can develop what’s known as nitrogen narcosis, which does an excellent job of mimicking the feeling of being drunk. The deeper you go, the drunker and more incapacitated you feel: Beyond 200 feet you might become acutely disoriented, at 300 feet you can black out. It’s not a good condition to be in when you’re in a place where you have to be calm, careful, and methodical if you want to survive. In addition, that amount of compressed oxygen becomes toxic to the human body. Around 1919, electronics engineer and inventor Elihu Thompson figured that divers could avoid nitrogen narcosis by breathing a mixture of helium and oxygen. In the following decades a gas cocktail called heliox was developed—mostly helium, with sufficient oxygen and maybe a little nitrogen. (Other breathing gases, such as trimix and nitrox, are also used by deep divers.)

Saturation divers breathe heliox for the entire time they are in storage. And this brings us back to those final family phone calls. Helium is about seven times lighter than air, and sound waves travel much more quickly through it. The result is that buff, often ex-military men performing deadly serious jobs end up sounding like cartoon characters—and not just for a few moments, but for weeks on end. In the unfortunately named BBC series Real Men, a saturation diver in storage calls his son to wish him a happy birthday. “It’s hard to understand my dad because he talks in a duck language,” the boy says later, “and I don’t speak duck.”

The divers and their support teams adjust pretty quickly to the vocal distortion, but it can still make communication tricky—especially when accents are involved. “A helium Geordie [a native of Newcastle] from the northeast of England in a team with a South African and a Belgian makes communicating quite a challenge,” Tweddle says. Tweddle, an affable, 39-year-old former police diver with a shaved head that takes on a gray cast by the end of a job, is the Geordie in the above scenario. Support vessels are usually equipped with a kind of descrambler for when the divers need to be in constant communication with the onboard support team, but the equipment is notoriously unreliable, and many dive supervisors choose not to use it.

When it’s time to enter the chamber (Hovey calls it the “house”), the divers pass through a tight, circular hatch at one end, like one might see on an old submarine, that closes with a “tunk.” The hatch is sealed, and even though they’re on a boat, just feet from support crew and fresh air, the divers might as well be on the International Space Station. Even farther actually: It takes about 3.5 hours for an astronaut to make it back from space. Saturation divers have to decompress for days at minimum. On a dive early in his career, when Hovey was on a job at a depth of 700 feet, he learned that his wife had miscarried. It would have taken him 11 days of decompression to exit the chamber. They needed his salary (not surprisingly, saturation divers are well-compensated, up to $1,400 per day), so his wife told him to finish the job.

The layouts of saturation systems differ from company to company, vessel to vessel, and oil field to oil field. Generally, the North Sea facilities tend to be slightly roomier, thanks to strict regulations, but that’s not to say they are in any way spacious; they are even the mildest claustrophobe’s worst nightmare. For Tweddle, entering that first hatch puts the him in a miniscule round room known as the “wet pot.” It is used to transfer the divers to the diving bell through a hatch in the ceiling—and it’s also the bathroom, with a tiny metal sink, toilet (more on that later), and showerhead. Through the wet pot, another hatch leads to the living space, where there is just enough room for four to six seats around a removable aluminum table. In other words, for up to six weeks, the divers will spend their waking hours either under hundreds of feet of water on the ocean floor or squeezed into an area the size of a booth at Applebee’s.

Beyond that, past a droopy blue curtain, is the sleeping area, with six double-stacked bunks squeezed into a “U” shape. The bunks at the far end are partially blocked by the others, and therefore particularly cramped (especially for six-foot-four-inch Tweddle). So bunk choice is a big deal. In the North Sea, divers draw lots. Among American divers, Hovey says, it’s seniority—or whoever gets there first. But at least divers do all get their own sleeping spaces. In the bad old days, Tweddle says, six guys would have to hot-bed three bunks, with one group working while the other slept. There are still ways to keep the work going 24/7. On some vessels, up to four saturation chambers can be linked together through side hatches in the wet pots, and connected by large metal hamster tubes. This enables multiple dive teams to be in constant rotation.

Once the divers are tightly ensconced in the saturation chamber, the life support crew begins pumping in heliox, and the “blowdown” begins. The time it takes to get fully pressurized depends on the depth of the work site. On this latest job, Hovey’s blowdown took a mere three hours. On another job, with a storage depth of 750 feet, blowdown was 10 hours. In essence, pressurization transforms the saturation chamber into a space in which the air around them—and filling their lungs and saturating their tissues—is exerting pressure equivalent to the weight of the water they will be working under. Getting pressed to 750 feet requires 333 psi or 22.66 atms. It means squeezing into the chamber 22 times the amount of air it would normally hold.

During the blowdown, the rapid increase in atmospheric pressure makes the chamber very hot and humid (fluid dynamics are so strange), and sometimes needs to be paused so the climate control system can catch up. Later, the thermostat will get turned up to 90 degrees because the poor thermal properties of helium leave the divers perpetually chilly. The divers fan themselves and work constantly to equalize their ears: yawning, swallowing, and using the Valsalva maneuver (the formal name for pinching your nose, closing your mouth, and blowing). The blowdown also leaves them achy for hours or even days. “Anything not liquid or solid is affected by the physics of the gas,” says Hovey. “The cartilage in your joints is porous and shrinks for a couple days. All your joints hurt or click with movement.”

Once the divers are at pressure, they can only try to get comfortable, and make the chamber something of a home. In truth, they don’t bring or need a lot of gear—just a few items of clothing, toiletries, magnets for attaching family pictures to the walls, reading material, some personal electronics, the occasional Star Wars pillowcase. Everything else can be sent in via one of the two airlocks—a porthole-sized one in the living quarters known as the medlock and a larger one, for things such as their dive suits, off the wet pot. An assistant life support technician (Hovey says all American divers he knows call this person a “sat Betty”) is available to collect dirty clothes and linens through the medlock. Books, tools, and dive logs also make the journey in and out regularly, with each transit taking a minute or two.

Four times a day, someone sends in a menu with meal options, which come from the ship’s mess in disposable to-go containers (more hygienic than reusable plates and cups). Not surprisingly, the food quality varies greatly by vessel and chef, but mostly it sounds like the bill of fare from a discount cruise. Steak, chicken, fish, mushy vegetables, and a salad bar including cheese and cold cuts. Mostly the divers see food as little more than fuel—they might consume up to 6,000 calories each day (more than double the recommended intake) to keep up with their demanding shifts in the water. They also take healthy doses of multivitamins, with an emphasis on vitamin D, to make up for the lack of sunlight. The food itself is unaffected by the pressure, but taste buds tend to get muted. Hot sauce is a popular personal item, but the divers must be sure to loosen the cap—otherwise the bottle of hot sauce (or shampoo, or clove oil in Hovey’s case) will implode during pressurization or explode during depressurization.

The toilet and shower are in the wet pot, and it is no easy feat to flush a toilet safely at that kind of pressure. There is a famous, unverifiable, and, we can only hope, apocryphal story about a diver whose buttocks created a seal with the toilet seat, so that when he opened a valve to flush, the pressure differential—well, it’s best not to say too much more, other than that toilet safety is taken very, very seriously. “The toilet must be filled halfway with water before use,” says Tweddle. “After use we request a flush, [a technician] opens a valve on surface, allowing us to operate two valves in a particular order to empty the toilet into the holding tank, then empty the tank into the ship’s wastewater system.” In other words, every flush is at least a two-person, multi-step job.

The six-man dive crews split into teams of two or three and alternate shifts. They are woken up an hour before they need to leave the ship. They eat and hydrate and use the bathroom (Tweddle says that one quality that makes a good saturation diver is “the ability to shit on command.”) They put on a layer of their own clothing, followed by their water-tight diving suits, which are equipped with circulating hot water systems to prevent hypothermia.

The dive team seals itself off in the wet pot and then, through the hatch in the ceiling, gets into a diving bell, at the same pressure as the chamber. Both spaces are sealed, and then they disconnect—what’s known as a “transfer under pressure.” Anytime seals are made or broken under that kind of extreme pressure there is the danger that a mishap could lead to an explosive decompression. One of saturation diving’s worst accidents occurred in 1983, when a dive bell was detached from the transfer hatch before it was completely sealed. Four divers and one dive technician were killed instantly and gruesomely. (Newer saturation systems have locking mechanisms to prevent this from happening.)

The bell, shaped like an egg and about the size of a shower stall, is crammed with gauges, switches, communication equipment, and loops of hoses, referred to as umbilicals, that carry gas, electricity, voice communication, hot water, and video feeds back and forth between the divers’ helmets and the ship, via the bell. Once the bell detaches from the chamber, it is guided over to a moonpool—a hole in the boat’s hull, essentially—where it is lowered by cable to the working depth. One diver stays in the bell to monitor breathing, hot water, communication, and electrical systems. The other diver (or other two) puts on a dive helmet (a “hat” to the divers) and departs out the bottom for six uninterrupted hours in the water. During that time, urinating isn’t an issue—it’s pretty much the only job in the world where it is expected that you will pee in your pants.

Hovey, on his Gulf assignment, was working to clear a hurricane-toppled, garbage-strewn platform dating to the 1970s. The job was supposed to take 14 days—52 days later work was suspended until the spring because of bad weather. Most of the work involved collecting massive amounts of junk—“anything bigger than a breadbox or smaller than a VW bus”—and placing it in large baskets that could be raised by the ship’s crane. There were old barrels and pipes and chemical tanks and stainless-steel banding material that comes in huge rolls but littered the area like 500-foot-long Slinkys. Divers carry minimal tools with them; everything else they need gets sent down directly from the ship—saws, torches, wrenches, welding equipment, collection bags.

Hovey and the others also needed to cut the platform’s original eight legs off, 20 feet below the mud. This required using a massive water jet to create ditches around them and then cutting through the legs with an oversized band saw—all while keeping an eye out for a mudslide in near-zero visibility. “Say an old diesel engine that’s halfway up your ditch wall finally starts working its way out,” Hovey says. “Everything kind of moves slow, so you’ll notice a little mud going past your leg and then all of a sudden there’s this huge bunch of pressure on your lower back and then it’s on your upper back and it’s starting to push you over and unless you climb onto the top of it you’ll definitely get buried.” Generally, the divers aim for neutral buoyancy, but they’re not swimming around with fins like recreational divers. They hop-walk like moon men, often in beat-up rubber boots.

Nobody works deep at the bottom of the ocean without an extensive background in commercial diving in shallower waters and lengthy, costly training in areas such as mixed-gas and closed-bell diving. Just signing up for those advanced courses requires a substantial amount of commercial dive time and even once one is fully certified, it can be hard to get hired for a saturation dive job without trusted people to vouch for your ability to work under those conditions without losing it. Despite all this effort, the occupation has a high attrition rate—though leaving the field isn’t always voluntary. Hovey guesses that of every 20 guys who graduate from a training program, maybe one is still doing the work after five years. Some leave because of the difficulty or long weeks away from home, but this is also a job that takes lives. There are not good statistics about saturation diving death rates, but a Centers for Disease Control and Prevention report from 1998 estimated that the occupational fatality rate for all commercial divers is 40 times the national average for other professions. Many divers have close calls that convince them that it’s not worth the risk.

Hovey recalls moments of deep panic. Once, his umbilicals got tangled up in a tool rack that had been lowered down. The movement of the boat above began to jerk him around and he worried the force was about to separate him from his gas supply. “You have to calm down, take some breaths, and say, ‘You are the only one that can help yourself. No one’s going to come down here and help you.’” Some divers have cheated death, with help from their partners, or through a combination of discipline, training, and luck, but usually when things go wrong, they go wrong very fast, with catastrophic consequences.

In 2016, Hovey was working in the Gulf of Mexico, replacing a long piece of vertical pipe. The crew attached a temporary support, a heavy rod they call a “strong back”—to keep the pipe from bending during installation. Because of a miscommunication, the strong back was released from the top before the bottom. It rolled around and, Hovey believes, crushed a diver—the one on the shift just after his—against the pipe. Back in the sat system, Hovey got word that there had been an accident. As attending medic, he readied first-aid items, but when the bell returned, he saw right away that the diver’s hat had been crushed. Protocol required Hovey and the other divers to perform 45 minutes of CPR. Then they respectfully moved his body to a hyperbaric lifeboat/decompression chamber available to all sat systems. The support team could do an accelerated, emergency decompression, since he was already dead. It still took two days.

Tweddle believes that all the crazy stuff he saw as a police diver (dead bodies) makes him pretty hard to rattle, but he’s had scary moments, too. In particular, there have been the times when he started breathing so hard that the system couldn’t keep up. It’s a perfect storm for panic, as you breathe an excess of carbon dioxide and begin to huff harder and harder. The only way out is to wrestle down the fear and breathe as slowly as possible until the system catches up. It’s so common a phenomenon that divers have a name for it, “Breathing past your hat.”

In addition to the physical demands, moving around large pieces of equipment in a medium the human body is not well suited to, the work requires incredible focus and is done in near-zero or zero (black water) visibility. The dive hats have lights, but that can make things worse when particles in the water scatter light around. The divers must be in constant communication with the topside dive supervisor, who guides the work, troubleshoots, and sends down equipment. But ultimately whether they succeed or fail at the job—and stay alive—is on the divers alone.

Much of Tweddle’s work these days is helping plug and cap wells. You might imagine that beneath each platform is a single well, but wells might be as far as five or six miles away from the platform, and they often come in clusters—imagine using 10 straws to drink a milkshake instead of just one. But now the North Sea oil run is winding down, as the price of oil stagnates and the cost of extraction from increasingly empty fields grows. New exploration has gravitated to areas too deep for even saturation divers. Soon, the jobs will be harder to come by.

Days in sat become a blur of work and rest and boredom, perhaps punctuated by fear. Sometimes, despite dynamic positioning systems that compensate for wind and swells, bad weather suspends work, and the vessel might even return to port without the divers knowing—until a strange face suddenly pops up outside one of the chamber’s portholes. The divers soon realize that they are being stared at like monkeys in a cage. Or, rather, monkeys in a small, pressurized metal tube.

The general rule for depressurization—desat—is 24 hours for each 100 feet of pressure. Those are some of the hardest days, the divers say, with no work to break the monotony and with the comforts of home—sunlight, big beds, privacy, home-cooked food, wives, kids—so close. Desat mostly takes place in the chamber, but Hovey has done two-man desats in the small hyperbaric lifeboat (which can keep the divers under pressure for 72 hours if the ship were to sink), so work can continue with a new team.

Even desat is stressful on the body. Divers report joint pain, headaches, shortness of breath. Experienced divers know the difference between these symptoms and the start of something more serious. On one job, Tweddle worked with a relatively inexperienced diver who felt panicky about his symptoms during desat. The only cure for early signs of the decompression sickness is to return to higher pressure, so the whole team had to start again. It’s imperative, in this job, to err on the side of safety.

When the divers finally come out of the chamber, the adjustment is both emotional and physical. They emerge pale and disoriented, like prisoners released from solitary, drained and irritable, body clocks out of whack. Tweddle finds it hard to train his body not to eat quite so much. He has to be on guard for waistline expansion since there are now strict body mass index guidelines for North Sea divers.

Hovey owns some land in the central Texas pine woods, and he usually spends a few days there alone before trying to reintegrate with the noise and chaos of family life. His kids give him a wide berth after a job, and he and his wife like to start dating all over again as a way to reconnect. It’s hard to shake the feeling that he is in suspended animation while in sat, even though life goes on. “My family is constantly trying to grow and be better versions of themselves,” he says. “Sometimes being away for work, I get left in the dust.”

But by the time the phone rings for the next job, both he and Tweddle will be ready. There’s something about the isolation and asceticism and discipline (and, of course, the money) that they are drawn to. Or maybe it’s the idea that they are working on the edges of human capability, facing danger with calm and planning, members of a club with very, very few members. They are, in so many ways, like astronauts. Only no one’s ever heard of them.

* Correction: This article was updated for the correct spelling of Steve Tweddle’s name, and to clarify, upon hearing from readers, that there may indeed be active female saturation divers.

BERLIN (Reuters) - German Radoslav Albrecht has founded an online bank that allows clients to transfer loans anywhere in the world using Bitcoin.

Bitbond uses cryptocurrencies like Bitcoin to bypass the Swift international transfer system to lend money across the globe rapidly and at low cost.

“Traditional money transfers are relatively costly due to currency exchange fees, and can take up to a few days,” Albrecht told Reuters TV in his office in Berlin’s fashionable neighborhood Prenzlauer Berg. “With Bitbond, payments work independently of where customers are. Via internet it is very, very quick and the fees are low.”

Clients hold the loans in digital tokens like Bitcoin only for seconds or minutes until they are exchanged back into the currency of the country where they wish to receive the funds, avoiding the crypto currencies fluctuating exchange rates.

Bitcoin has been used as collateral for loans, but never as a way of transferring credit in currency internationally.

Albrecht’s service has been growing in popularity among clients since he launched the company in 2013. His office employs 24 people from 12 countries who manage loans for 100 clients amounting to around $1 million each month.

Most clients are small business owners or freelance workers, Albrecht says. Loans are relatively small and don’t exceed 50,000 dollars. In 2016, Bitbond was officially licensed as a bank and has gained many investors since.

Adoption of Bitcoin has been rapid in Germany. It trails only the U.S., according to Bitnodes, which tracks the location of all the Bitcoin nodes that transmit data about new transactions.

I have been job hunting in the last 2 months or so. Thankfully, I am settled with a great choice now, and it seemed like I had more luck than the average job applicant. But among the handful of companies that I had the pleasure to interview with, I had my fair share of interesting moments.

The very first interview I had with a startup company was one of them. I mentioned in the resume I sent that I made an distro for the Raspberry Pi called Crankshaft, which is related to the work that they are doing and got a phone interview appointment. We chatted for a bit to introduce ourselves to each other. After 30 minutes, I was asked that whether I was in front of a computer (implying it will be a live coding exercise) out of the blue. To set the record straight, I'm totally happy to do live coding exercises, and I understand companies that are fed up with impostors who "inflated" their experiences on their resume. However, in that instance, there was no indication that it was coming even in the interview appointment email. Nonetheless, I figured I can swallow it. After all, I have been programming for more than a half of my life. I figured being able to code spontaneously shows that I am a fearless ninja warrior when it comes to programming :^).

I was presented with a Leetcode-esque question to which I said, "I can do it in two commands chained by a pipe in bash." And he said, let's treat it like a real CS question. So I asked, what languages can I code in, and the answer was any programming language I like. So I chose to program in Go. Now, when it comes to live coding challenges, one of the biggest problems is that the interviewer always knows the optimal answer when the applicant most likely can only come up with a suboptimal one given a very short time allowance. In a relatively successful interview, either the optimal solution is found when the two parties work on the plan of attack before the coding phase, or in the Q&A phase when a potential question or hint is presented to the interviewee. I presented the general direction I would take. In this case, I did not come up with the optimal algorithm but nonetheless, we agreed to proceed to the coding phase. In the Q&A session, the interviewer asked me why did I choose Go but not C. My answer was I would always choose Go over C if memory or performance is not a required constraint. Go is easier to work with and less boilerplate to worry about. Then, the interviewer mentioned something about a solution involving hashes that makes the lookup operation O(1) and I disagreed with him, so then we settled on them being "practically O(1)." Finally, we talked about some other stuff and I could sense that the interviewer being disappointed that I didn't come up with the optimal answer. I answered something like "I honestly don't know the answer for that – there are more things that I don't know than things I do know." He asked, "But didn't you write an embedded OS?". Then I realized that he was expecting me to be using C and coming up with the most optimal answer, and not taking "I don't know" for an answer because I wrote an embedded OS. So I told him "Oh, it is actually written in bash, you can see it on Github," and I can feel that he wasn't very impressed by my leet bash programming skillz. The interview concluded not long after that. I sent him a thank-you email but never heard back.

I guess I didn't qualify for the next round and was eliminated straight out. I don't intend to write this post to bash the interviewer or to say I should have been qualified – I didn't and I deserved it. I think though, there is something to say about the virtue of having, expecting, and imposing artificial constraints when it comes to problem-solving in general.

Truth to be told, more than 95% of Crankshaft code is written in bash. I am not even a leet bash programmer. I still have to look up "bash equality syntax" when I need to do a number comparison because the syntax confuses me (I remember the string equality syntax now, though).

That brings me back to the past. After dropping out of college in Vietnam years ago, I had the pleasure to work on a much less technically-demanding job in a small company that resells various specialized software packages. I had the chance to interact a lot with the CEO and the CTO. I first heard the quote "if all you have is a hammer, everything looks like a nail" from the CTO, which has since become one of my favorite ones. The CEO told me he studied Vovinam, a Vietnamese martial art. He told me that one of the main themes of Vovinam is using "soft" to win over "hard." Essentially, it means whenever possible, one should exert the least energy, basically choose the path of least resistance to achieve what they want to do. In overall, I learned the spirit from both of them: It is important to first think about the goal rather than the mean.

The trend of using hard technology is prevalent in tech-savvy people. Just today, someone asked on a forum about using their phone to VNC to their Raspberry Pi to watch Youtube videos because they are inspired by the lack of Adblock on the Youtube app on mobile. They started worrying about Bluetooth and battery constraints, and VNC, and using Bluetooth, and bandwidth and interference between Bluetooth and Wifi. Which is indeed solvable by googling "youtube client without ads" and you'll see NewPipe among the top results, which is an app you can install in 30 seconds to solve the problem. But when you have a hammer that is the Raspberry Pi, the Adblock problem starts to look like a nail.

Recently, a close friend of mine who is a founder of a small company back home asked me to advise him on making a website/app to have "Uber for X" (X being his non-tech business). Upon talking, I realized if he wants to compete by making a good website/app, it's both hard and expensive to do. If his tech model turns out to be relatively successful, a medium-sized company in tech will be able to easily copy his model and out-compete him. Perhaps all he wants to do only need a website to advertise but he doesn't need a website or an app to execute. All he needs is an Excel spreadsheet and a relatively competent secretary to manage that spreadsheet and call people with demand and supply. But when you have a hammer that is Uber, the supply-and-demand problem starts to look like a nail, even when you don't have millions of people using it in real-time.

And I see the same trend in Machine learning, Big data, AI, Cryptocurrency, Internet of Things, Social networks, Decentralized networks, etc. Everything in life, however trivial or hard, is a nail that can be solved with those fancy hammers. So we can have a tablet juxtaposed on a fridge, a bluetooth speaker integrated to a salt shaker, a subscription model for juicers, and machine learning for all kinds of social problems.

I enjoy hard problems a lot. In the last couple of years, working on CUDA kernels to do parallel string comparison is one of the topics I busted my chops on. Some problems are extremely hard and need really clever algorithms and tuning to solve. But alongside with those, there are problems that only need a generally good idea to solve with very little hard code required.

And that's the moment of enlightenment: Writing dumb scripts may become my niche as a person who "wrote an embedded OS."

By now, the emails will be incredibly familiar. They have the same chirpy tone, are being sent from brands and ask whether you'd like to get more emails from that company. (It's likely you completely forgot what the company was, let alone what you purchased from it in 2007).

"We're committed to managing and safeguarding the information you give us when looking for a job," reads a typical one from a recruitment website. "CLICK HERE TO STAY SIGNED UP," shouts another before continuing: "We don’t want to lose you, so please take action now". Some have even claimed user accounts will have to be deleted if a reply isn't recieved.

The majority of these emails cite the European General Data Protection Regulation (GDPR), which starts to be enforced on May 25. GDPR introduces changes to how businesses and organisations should handle personal information – and for companies faced with the prospect of huge fines for breaching the new rules, it's causing panic. And that's why you're getting all those emails.

"We’ve heard stories of email inboxes bursting with long emails from organisations asking people if they’re still happy to hear from them," Steve Wood, the deputy information commissioner for the UK wrote in a blog post earlier this week. "Think about whether you actually need to refresh consent before you send that email and don’t forget to put in place mechanisms for people to withdraw their consent easily," Wood says.

But, it turns out, most of these emails are pointless. "In the UK it has been the law since 2003 that you can only send a marketing email to an individual recipient when they have consented to receive it or you have an existing customer relationship with them and have offered them the opportunity to opt out," explains Jon Baines, data protection advisor at law firm Mishcon de Reya.

So why are they sending these emails? It's largely around the fear of GDPR. The regulation says companies can be fined up to €20 million or four per cent of their annual global turnover. Many companies are keen to get their systems in order. Although in the UK the Information Commissioner has made it clear it won't be heavy-handed with fines.

Baines believes a big reason why these emails are being sent at the moment is because of an "increased awareness around the fact that sending marketing emails requires either the consent of the recipient or an existing customer relationship". That awareness has been amplified because of the hype around GDPR.

However, the Privacy and Electronic Communications (EC Directive) Regulations – known as PECR for short – govern marketing messages. These are based upon a European e-privacy Directive and cover messages used for marketing – everything from the pesky emails to text messages.

GDPR doesn't replace PECR but sits alongside it and European regulators are coming up with a new set of e-privacy rules to replace it. Confused? So are the companies emailing you. The result is a slightly messy mix of rules: both GDPR and PECR are dense, legally complex and have a plethora of caveats with exemptions for different scenarios.

But the existence of PECR means that in a large amount of cases, companies may not have necessarily needed to send the emails re-asking for permission to keep in touch. "I think a lot of the emails people are receiving are unnecessary, because people have either already consented or are receiving them to business addresses," Baines says. Business email addresses – for instance, rowland@wired.co.uk – fall under GDPR as personal data, but for marketing messages consent to receive them may not be needed.

If people haven't already consented to receive marketing messages, the company sending them will have been in breach of PECR, potentially for many years.

But what is considered consent is a slightly murky affair. With the introduction of GDPR comes an updated definition of what consent is. It's complex but states consent has to be unambiguous and involve someone actively saying yes. For instance, a pre-ticked box saying you are willing to receive marketing emails doesn't count as unambiguous consent. But a box you have to actively tick does.

"If consent is the appropriate lawful basis then that energy and effort must be spent establishing informed, active, unambiguous consent," the ICO's Wood says. As well as consent, there are other ways for companies to obtain and process a person's data and still be inline with the requirements of GPDR.

Ultimately, the overlap between PECR and GDPR has meant some companies will lose subscribers to their mailing lists that have just ignored the deluge of messages being received. In an almost ironic twist, last year the ICO fined Honda and Flybe for sending emails asking people to agree to getting more emails. "Sending emails to determine whether people want to receive marketing without the right consent, is still marketing and it is against the law," the ICO said at the time.

But there also have been more malicious examples of email consent messages being sent. UK-based cybersecurity firm Redscan discovered phishing emails have been sent that were disguised as GDPR-related emails. The firm spotted a fake email that had been made to look like it was from Airbnb, stating its customers should click on a particular link to update their privacy settings.

When clicked, the link would take users to a spoofed Airbnb website that collected all the details entered and saved them to systems belonging to the hackers that created the website. “The irony won’t be lost on anyone that cybercriminals are exploiting the arrival of new data protection regulations to steal people’s data,” Redscan's director of cybersecurity Mark Nicholls, said in a statement.

I’ve recently been writing a big-integer library, zig-bn in the Zig programming language.

The goal is to have reasonable performance in a fairly simple implementation with a generic implementation with no assembly routines.

I’ll list a few nice features about Zig which I think suit this sort of library before exploring some preliminary performance comparisons and what in the language encourages the speed.

Transparent Local Allocators

Unlike most languages, the Zig standard library does not have a default allocator implementation. Instead, allocators are specified at runtime, passed as arguments to parts of the program which require it. I’ve used the same idea with this big integer library.

The nice thing about this is it is very easy to use different allocators on a per-integer level. A practical example may be to use a faster stack-based allocator for small temporaries, which can be bounded by some upper limit.

// Allocate an integer on the heap
var heap_allocator = std.heap.c_allocator;
var a = try BigInt.init(heap_allocator);
defer a.deinit();

// ... and one on the stack
var stack_allocator = std.debug.global_allocator;
var b = try BigInt.init(stack_allocator);
defer b.deinit();

// ... and some in a shared arena with shared deallocation
var arena = ArenaAllocator.init(heap_allocator);
defer arena.deinit();

var c = try BigInt.init(&arena.allocator);
var d = try BigInt.init(&arena.allocator);

This isn’t possible in GMP, which allows specifying custom allocation functions, but which are shared across the all objects. Only one set of memory functions can be used per program.

Handling OOM

One issue with GMP is that out-of-memory conditions cannot easily be handled. The only feasible way in-process way is to override the allocation functions and use exceptions in C++, or longjmp back to a clean-up function which can attempt to handle this as best as it can.

Since Zig was designed to handle allocation in a different way to C, we can handle these much more easily. For any operation that could fail (either out-of-memory or some other generic error), we can handle the error or pass it back up the call-stack.

var a = try BigInt.init(failing_allocator);
// maybe got an out-of-memory! if we did, lets pass it back to the caller
try a.set(0x123294781294871290478129478);

There is the small detriment that it is required to explicitly handle possible failing functions (and for zig-bn, that is practically all of them). The provided syntax makes this minimal boilerplate, and unlike GMP we can at least see where something could go wrong and not have to rely on hidden error control flow.

Compile-time switch functions

Zig provides a fair amount of compile-time support. A particular feature is the ability to pass an arbitrary type var to a function. This gives a duck-typing sort of feature and can provide more fluent interfaces than we otherwise could write.

For example:

pub fn plusOne(x: var) @typeOf(x) {
    const T = @typeOf(x);

    switch (@typeInfo(T)) {
        TypeId.Int => {
            return x + 1;
        },
        TypeId.Float => {
            return x + 1.0;
        },
        else => {
            @compileError("can't handle this type, sorry!");
        },
    }
}

This feature is used to combine set functions into a single function instead of needing a variety of functions for each type as in GMP (mpz_set_ui,mpz_set_si, …).

Peformance

Perhaps the most important detail of a big integer library is its raw performance. I’ll walk through the low-level addition routine and look at some techniques we can use to speed it up incrementally.

The benchmarks used here can be found in this repository. We simply compute the 50000’th fibonacci number. This requires addition and subtraction only.

Our initial naive implementation is as follows. It uses 32-bit limbs (so our double-limb is a 64-bit integer) and simply propagates the carry. We force inline the per-limb division and our debug asserts are compiled out in release mode. Memory allocation is handled in the calling function.

// a + b + *carry, sets carry to overflow bits
fn addLimbWithCarry(a: Limb, b: Limb, carry: &Limb) Limb {
    const result = DoubleLimb(a) + DoubleLimb(b) + DoubleLimb(*carry);
    *carry = @truncate(Limb, result >> Limb.bit_count);
    return @truncate(Limb, result);
}

fn lladd(r: []Limb, a: []const Limb, b: []const Limb) void {
    debug.assert(a.len != 0 and b.len != 0);
    debug.assert(a.len >= b.len);
    debug.assert(r.len >= a.len + 1);

    var i: usize = 0;
    var carry: Limb = 0;

    while (i < b.len) : (i += 1) {
        r[i] = @inlineCall(addLimbWithCarry, a[i], b[i], &carry);
    }

    while (i < a.len) : (i += 1) {
        r[i] = @inlineCall(addLimbWithCarry, a[i], 0, &carry);
    }

    r[i] = carry;
}

The results are as follows:

fib-zig: 0:00.75 real, 0.75 user, 0.00 sys
  debug: 0:06.61 real, 6.60 user, 0.00 sys

For comparison, the GMP run time is:

fib-c:   0:00.17 real, 0.17 user, 0.00 sys

A more comparable C implementation (python) is:

fib-py:  0:00.77 real, 0.77 user, 0.00 sys

A bit of work to do against GMP! We aren’t out of the ballpark compared to less heavily optimized libraries. We are comparing the debug runtime version as well since I consider it important that it runs reasonably quick for a good development cycle, and not orders of magnitude slower.

Leveraging Compiler Addition Builtins

Zig provides a number of LLVM builtins to us. While these shouldn’t usually be required, they can be valuable in certain cases. We’ll be using the @addWithOverflow builtin to perform addition while catching possible overflow.

Our new addition routine is now:

fn lladd(r: []Limb, a: []const Limb, b: []const Limb) void {
    debug.assert(a.len != 0 and b.len != 0);
    debug.assert(a.len >= b.len);
    debug.assert(r.len >= a.len + 1);

    var i: usize = 0;
    var carry: Limb = 0;

    while (i < b.len) : (i += 1) {
        var c: Limb = 0;
        c += Limb(@addWithOverflow(Limb, a[i], b[i], &r[i]));
        c += Limb(@addWithOverflow(Limb, r[i], carry, &r[i]));
        carry = c;
    }

    while (i < a.len) : (i += 1) {
        carry = Limb(@addWithOverflow(Limb, a[i], carry, &r[i]));
    }

    r[i] = carry;
}

The new results:

fib-zig: 0:00.69 real, 0.69 user, 0.00 sys
  debug: 0:06.47 real, 6.42 user, 0.00 sys

A minimal, but noticeable improvement.

Improving Debug Performance

Debug mode in Zig performs runtime bounds checks which include array checks and other checks for possible undefined behavior.

For these inner loops this is a lot of overhead. Our assertions are sufficient to cover all the looping cases. We can disable these safety checks on a per-block basis:

fn lladd(r: []Limb, a: []const Limb, b: []const Limb) void {
    @setRuntimeSafety(false);
    ...
}

fib-zig: 0:00.69 real, 0.69 user, 0.00 sys
  debug: 0:03.91 real, 3.90 user, 0.00 sys

That is a lot better.

64-bit limbs (and 128-bit integers).

We have been using 32-bit words this entire time. Our machine word-size however is 64-bits. Lets change our limb size only, and rerun our tests.

fib-zig: 0:00.35 real, 0.35 user, 0.00 sys
  debug: 0:01.95 real, 1.95 user, 0.00 sys

Unsurprisingly, this is now twice as fast! It is fairly useful if your compiler supports builtin 128-bit integer types when using 64-bit limbs. The reason is it makes handling overflow in addition and especially multiplication much more simple and easier to optimize by the compiler. Otherwise, software workarounds need to be done which can be much less performant.

Implementation Performance Summary

Benchmark code here.

A performance comparison using the following libraries/languages:

Note that C and Go use assembly, while Rust/CPython both are implemented in Rust and C respectively, and are comparable as non-tuned generic implementations.

System Info

Architecture:        x86_64
Model name:          Intel(R) Core(TM) i5-6500 CPU @ 3.20GHz

Compiler Versions

zig:  0.2.0.ef3111be
gcc:  gcc (GCC) 8.1.0
go:   go version go1.10.2 linux/amd64
py:   Python 3.6.5
rust: rustc 1.25.0 (84203cac6 2018-03-25)

Addition/Subtraction Test

Computes the 50,000th fibonacci number.

fib-zig: 0:00.35 real, 0.35 user, 0.00 sys
fib-c:   0:00.17 real, 0.17 user, 0.00 sys
fib-go:  0:00.20 real, 0.20 user, 0.00 sys
fib-py:  0:00.75 real, 0.75 user, 0.00 sys
fib-rs:  0:00.81 real, 0.81 user, 0.00 sys

Multiplication/Addition Test

Computes the 50,000th factorial.

Zig uses naive multiplication only while all others use asymptotically faster algorithms such as (karatsuba multiplication)[https://en.wikipedia.org/wiki/Karatsuba_algorithm].

fac-zig: 0:00.54 real, 0.54 user, 0.00 sys
fac-c:   0:00.18 real, 0.18 user, 0.00 sys
fac-go:  0:00.21 real, 0.21 user, 0.00 sys
fac-py:  0:00.50 real, 0.48 user, 0.02 sys
fac-rs:  0:00.53 real, 0.53 user, 0.00 sys

Division Test (single-limb)

Computes the 20,000th factorial then divides it back down to 1.

Rust is most likely much slower since it doesn’t special-case length 1 limbs.

facdiv-zig: 0:00.99 real, 0.98 user, 0.00 sys
facdiv-c:   0:00.16 real, 0.16 user, 0.00 sys
facdiv-go:  0:00.93 real, 0.93 user, 0.00 sys
facdiv-py:  0:00.99 real, 0.99 user, 0.00 sys
facdiv-rs:  0:05.01 real, 4.98 user, 0.00 sys

Summary

In short, zig-bn has managed to get fairly good performance from a pretty simple implementation. It is twice as fast as other generic libraries for the functions we have optimized, and is likely to be similarly fast using comparable algorithms for multiplication/division.

While I consider these good results for a very simple implementation (<1k loc, excluding tests) it is still lacking vs. GMP. Most notably, the algorithms used are much more advanced and the gap would continue to grow as numbers grew even larger. Hats off to the GMP project, as always.

A good start for a weeks work.

While learning the Nim language and trying to correlate that with my Python 3 knowledge, I came across this awesome comparison table ofstring manipulation functions between the two languages.

My utmost gratitude goes to the developers of Nim, Python, Org,ob-nim and ob-python, and of course Hugo which allowed me to publish my notes in this presentable format.

Here are the code samples and their outputs. In each mini-section below, you will find a Python code snippet, followed by its output, and then the same implementation in Nim, followed by the output of that.

The tool versions used are:

• Python 3.6.2
• Nim 0.18.1 [Linux: amd64] git hash: 50039422 (devel branch)

Updates #

<2017-12-13 Wed>

Update the Nim snippets output using the improvedecho! The echo output difference is notable in the .split examples. This fixes the issue about confusing echo outputs that I raised in Nim Issue #6225. Big thanks toFabian Keller for Nim PR #6825!

<2017-11-29 Wed>

Update the Understanding the ^N syntax example that gave incorrect output before Nim Issue #6223 got fixed.

<2017-11-28 Tue>

Update the .join example that did not work before Nim Issue #6210 got fixed.

<2017-11-27 Mon>

Use the binary operator ..< instead of the combination of binary operator .. and the deprecated unary < operator.

String slicing #

All characters except last #

str="a\tbc\tdef\taghij\tcklm\tdanopqrstuv\tadefwxyz\tzyx"print(str[:-1])

a	bc	def	aghij	cklm	danopqrstuv	adefwxyz	zy

varstr="a\tbc\tdef\taghij\tcklm\tdanopqrstuv\tadefwxyz\tzyx"# Always add a space around the .. and ..< operatorsechostr[0..<str.high]# orechostr[0..^2]# orechostr[..^2]

a	bc	def	aghij	cklm	danopqrstuv	adefwxyz	zy
a	bc	def	aghij	cklm	danopqrstuv	adefwxyz	zy
a	bc	def	aghij	cklm	danopqrstuv	adefwxyz	zy

Understanding the ^N syntax #

varstr="abc"# Always add a space around the .. and ..< operatorsecho"1st char(0) to last, including \\0(^0) : ",str[0..^0]# Interestingly, this also prints the NULL character in the output.. looks like "abc^@" in Emacsecho"1st char(0) to last       (^1) «3rd»  : ",str[0..^1]echo"1st char(0) to 2nd-to-last(^2) «2nd»  : ",str[0..^2]echo"1st char(0) to 3rd-to-last(^3) «1st»  : ",str[0..^3]echo"1st char(0) to 4th-to-last(^4) «0th»  : ",str[0..^4]# echo "1st char(0) to 4th-to-last(^4) «0th»  : ", str[0 .. ^5] # Error: unhandled exception: value out of range: -1 [RangeError]# echo "2nd char(1) to 4th-to-last(^4) «0th»  : ", str[1 .. ^4] # Error: unhandled exception: value out of range: -1 [RangeError]echo"2nd char(1) to 3rd-to-last(^3) «1st»  : ",str[1..^3]echo"2nd char(1) to 2nd-to-last(^2) «2nd»  : ",str[1..^2]echo"2nd char(1) to last,      (^1) «3rd»  : ",str[1..^1]echo"Now going a bit crazy .."echo" 2nd-to-last(^2) «2nd» char to 3rd(2)         : ",str[^2..2]echo" 2nd-to-last(^2) «2nd» char to last(^1) «3rd» : ",str[^2..^1]echo" 3rd-to-last(^3) «1st» char to 3rd(2)         : ",str[^3..2]

1st char(0) to last, including \0(^0) : abc
1st char(0) to last       (^1) «3rd»  : abc
1st char(0) to 2nd-to-last(^2) «2nd»  : ab
1st char(0) to 3rd-to-last(^3) «1st»  : a
1st char(0) to 4th-to-last(^4) «0th»  :
2nd char(1) to 3rd-to-last(^3) «1st»  :
2nd char(1) to 2nd-to-last(^2) «2nd»  : b
2nd char(1) to last,      (^1) «3rd»  : bc
Now going a bit crazy ..
 2nd-to-last(^2) «2nd» char to 3rd(2)         : bc
 2nd-to-last(^2) «2nd» char to last(^1) «3rd» : bc
 3rd-to-last(^3) «1st» char to 3rd(2)         : abc

Notes #

• It is recommended to always use a space around the .. and ..< binary operators to get consistent results (and no compilation errors!). Examples: [0 ..< str.high], [0 .. str.high], [0 .. ^2],[ .. ^2]. This is based on the tip by Andreas Rumpf (also one of the core devs of Nim). You will find the full discussion around this topic of dots and spaces in Nim Issue #6216.

  Special ascii chars like %.&$ are collected into a single operator token. – Araq

• To repeat: Always add a space around the .. and ..< operators.

• As of commit 70ea45cdba, the < unary operator is deprecated! So do [0 ..< str.high] instead of [0 .. <str.high] (see Nim Issue #6788).

• With the example str variable value being "abc", earlier bothstr[0 .. ^5] and str[1 .. ^4] returned an empty stringincorrectly! (see Nim Issue #6223). That now got fixed in commit b74a5148a9. After the fix, those will cause this error:

  system.nim(3534)         []
  system.nim(2819)         sysFatal
  Error: unhandled exception: value out of range: -1 [RangeError]

  Also after this fix, str[0 .. ^0] outputs abc^@ (where ^@ is the representation of NULL character).. very cool!

All characters except first #

str="a\tbc\tdef\taghij\tcklm\tdanopqrstuv\tadefwxyz\tzyx"print(str[1:])

	bc	def	aghij	cklm	danopqrstuv	adefwxyz	zyx

varstr="a\tbc\tdef\taghij\tcklm\tdanopqrstuv\tadefwxyz\tzyx"# echo str[1 .. ] # Does not work.. Error: expression expected, but found ']'# https://github.com/nim-lang/Nim/issues/6212# Always add a space around the .. and ..< operatorsechostr[1..str.high]# orechostr[1..^1]# second(1) to last(^1)

	bc	def	aghij	cklm	danopqrstuv	adefwxyz	zyx
	bc	def	aghij	cklm	danopqrstuv	adefwxyz	zyx

All characters except first and last #

str="a\tbc\tdef\taghij\tcklm\tdanopqrstuv\tadefwxyz\tzyx"print(str[1:-1])

	bc	def	aghij	cklm	danopqrstuv	adefwxyz	zy

varstr="a\tbc\tdef\taghij\tcklm\tdanopqrstuv\tadefwxyz\tzyx"# Always add a space around the .. and ..< operatorsechostr[1..<str.high]# orechostr[1..^2]# second(1) to second-to-last(^2)

	bc	def	aghij	cklm	danopqrstuv	adefwxyz	zy
	bc	def	aghij	cklm	danopqrstuv	adefwxyz	zy

Count #

str="a\tbc\tdef\taghij\tcklm\tdanopqrstuv\tadefwxyz\tzyx"print(str.count('a'))print(str.count('de'))

importstrutilsvarstr="a\tbc\tdef\taghij\tcklm\tdanopqrstuv\tadefwxyz\tzyx"echostr.count('a')echostr.count("de")

Starts/ends with #