“I was struck by the exceptional magnitude of similarity among friends,” said Carolyn Parkinson, a cognitive scientist at the University of California, Los Angeles. The results “were more persuasive than I would have thought.” Dr. Parkinson and her colleagues, Thalia Wheatley and Adam M. Kleinbaum of Dartmouth College, reported their results in Nature Communications.

“I think it’s an incredibly ingenious paper,” said Nicholas Christakis, author of “Connected: The Power of Our Social Networks and How They Shape Our World” and a biosociologist at Yale University. “It suggests that friends resemble each other not just superficially, but in the very structures of their brains.”

The findings offer tantalizing evidence for the vague sense we have that friendship is more than shared interests or checking off the right boxes on a Facebook profile. It’s about something we call good chemistry.

“Our results suggest that friends might be similar in how they pay attention to and process the world around them,” Dr. Parkinson said. “That shared processing could make people click more easily and have the sort of seamless social interaction that can feel so rewarding.”

Kevin N. Ochsner, a cognitive neuroscientist at Columbia University who studies social networks, said the new report is “cool,” “provocative” and “raises more questions than it answers.” It could well be picking up traces of “an ineffable shared reality” between friends.

Dr. Ochsner offered his own story as evidence of the primacy of chemistry over mere biography. “My wife-to-be and I were both neuroscientists in the field, we were on dating websites, but we were never matched up,” he said.

“Then we happened to meet as colleagues and in two minutes we knew we had the kind of chemistry that breeds a relationship.”

Dr. Parkinson — who is 31, wears large horn-rimmed glasses and has the wholesome look of a young Sally Field — described herself as introverted but said, “I’ve been fortunate with my friends.”

The new study is part of a surge of scientific interest in the nature, structure and evolution of friendship. Behind the enthusiasm is a virtual Kilimanjaro of demographic evidence that friendlessness can be poisonous, exacting a physical and emotional toll comparable to that of more familiar risk factors like obesity, high blood pressure, unemployment, lack of exercise, smoking cigarettes.

Scientists want to know what, exactly, makes friendship so healthy and social isolation so harmful, and they’re gathering provocative, if not yet definitive, clues.

Dr. Christakis and his co-workers recently demonstrated that people with strong social ties had comparatively low concentrations of fibrinogen, a protein associated with the kind of chronic inflammation thought to be the source of many diseases. Why sociability might help block inflammation remains unclear.

Researchers have also been intrigued by evidence of friendship among nonhuman animals, and not just in obvious candidates like primates, dolphins and elephants.

Gerald G. Carter of the Smithsonian Tropical Research Institute in Panama and his colleagues reported last year that female vampire bats cultivate close relationships with unrelated females and will share blood meals with those friends in harsh times — a lifesaving act for animals that can’t survive much more than a day without food.

Through years of tracking the behaviors of a large flock of great tits, Josh A. Firth of Oxford University and his co-workers found that individual birds showed clear preferences for some flock members over others. When a bird’s good friend died or disappeared, the bereft tit began making overtures to other birds to replace the lost comrade.

Yet when it comes to the depth and complexity of bonds, humans have no peers. Dr. Parkinson and her co-workers previously had shown that people are keenly and automatically aware of how all the players in their social sphere fit together, and the scientists wanted to know why some players in a given network are close friends and others mere nodding acquaintances.

Inspired by the research of Uri Hasson of Princeton, they decided to explore subjects’ neural reactions to everyday, naturalistic stimuli — which these days means watching videos.

The researchers started with a defined social network: an entire class of 279 graduate students at an unnamed university widely known among neuroscientists to have been the Dartmouth School of Business.

The students, who all knew one another and in many cases lived in dorms together, were asked to fill out questionnaires. Which of their fellow students did they socialize with — share meals and go to a movie with, invite into their homes? From that survey the researchers mapped out a social network of varying degrees of connectivity: friends, friends of friends, third-degree friends, friends of Kevin Bacon.

The students were then asked to participate in a brain scanning study and 42 agreed. As an fMRI device tracked blood flow in their brains, the students watched a series of video clips of varying lengths, an experience that Dr. Parkinson likened to channel surfing with somebody else in control of the remote.

They watched astronaut Chris Hadfield demonstrate how water behaves like a goopy gel in low gravity. They watched a sedately sentimental scene from a Jewish wedding between two people who happened to be gay men.

They watched the author Eric Schlosser warn of the dangers of allowing a few fast-food giants to control our food supply. They watched what my good friend Judy Gradwohl and I agreed, on reviewing the clips together later, was one of the worst music videos ever produced, about a man with an obviously fake facial deformity who is bullied at work and snubbed by his attractive female colleague but who eventually wins her heart when the bullies turn on her and he, Phony Elephant Man, steps in and beats them up.

The students watched pratfall comedy clips and an Australian mockumentary so subtle that certain viewers confessed they didn’t realize it was a spoof but liked it nonetheless.

Analyzing the scans of the students, Dr. Parkinson and her colleagues found strong concordance between blood flow patterns — a measure of neural activity — and the degree of friendship among the various participants, even after controlling for other factors that might explain similarities in neural responses, like ethnicity, religion or family income.

The researchers identified particularly revealing regions of pattern concordance among friends, notably in the nucleus accumbens, in the lower forebrain, which is key to reward processing, and in the superior parietal lobule, located toward the top and the back of the brain — roughly at the position of a man bun — where the brain decides how to allocate attention to the external environment.

Using the results, the researchers were able to train a computer algorithm to predict, at a rate well above chance, the social distance between two people based on the relative similarity of their neural response patterns.

Dr. Parkinson emphasized that the study was a “first pass, a proof of concept,” and that she and her colleagues still don’t know what the neural response patterns mean: what attitudes, opinions, impulses or mental thumb-twiddling the scans may be detecting.

They plan next to try the experiment in reverse: to scan incoming students who don’t yet know one another and see whether those with the most congruent neural patterns end up becoming good friends.

Alexander Nehamas, a professor of philosophy at Princeton University and author of a meditative book, “On Friendship,” appreciated the design of the study and its use of video clips to ferret out the signature of friendship.

“The aesthetic choices we make, the things we like, the taste we have in art, plays, TV, furniture — when you put them together they are absolutely essential components of our character, an indication of who we are,” he said. We live “immersed in art.”

Not high art, not a night-at-the-opera art, but everyday art — buildings, billboards, clothing, the dishes at a restaurant, the percussive rhythms of subways on train tracks.

“Watching TV clips is much more accurate to our everyday life than the times we go to a museum,” he said, and therefore potentially more revealing of who we are and what we hope to find in a friend.

So if you happened to catch “The Cute Show: Sloths!,” about a self-proclaimed “sloth sanctuary” in Costa Rica, and if your first thought wasn’t ooh, how adorable those little smiley sloths are, but rather, sloths are not pets to be cuddled and don’t bathe the algae off their fur — haven’t you heard of mutualism? — give me a call.

We’ll be biosnob soul mates for life.

README.md

I have a vehement dislike of spam. Right there, that's something you and I have in common because I'm yet to meet a person who says "well actually, I find those Viagra emails I receive every day kinda useful". We get bombarded by spam on a daily basis and quite rightly, people get kinda cranky when they have to deal with it; it's an unwanted invasion that takes a little slice of unnecessary mental processing each time we see it. Sure, junk mail filters catch a lot of it, but even the best implementations will still let a few slip through now and then.

But for the most part, spam is indiscriminate; you're on a list so bam - you get emails about improving your bedroom life. Increasingly though, it's much more targeted than that and is invading channels beyond just email:

Now just to avoid any shadows of doubt, yes, this meets the very definition of spam:

the use of electronic messaging systems to send an unsolicited message (spam), especially advertising, as well as sending messages repeatedly on the same site

You'll also see the Twitter example above referred to as "thread-hijacking", that is someone interjecting in a discussion in order to divert it in a different direction. In the case above, HostGator was being taken to task for storing passwords in a retrievable fashion (i.e. not as a strong cryptographic hash), and rightly so too. The hijacking then came via Nodestack who decided that this was the perfect opportunity to try and sell their services.

I'm going to talk more about Nodestack's behaviour in a moment, but let me clear about this - spam and thread-hijacking in this fashion is all too common:

It's the same deal - targeted spam. Someone at SignNow had decided that Mister Johnson met their target demographic and thus deserved an unsolicited advertisement. Don't confuse this with a Twitter ad either; we as users of the free platform know that the trade-off is companies pay to highlight their tweets, each of which is clearly flagged as "promoted". SignNow's approach was likely highly automated based on keywords in Mister Johnson's tweet rather than them thinking he genuinely needed their services - there's very little likelihood that someone complaining about phishing emails impersonating DocuSign is going to turn around and then start using an alternative electronic signature platform. It was probably just a case of him mentioning @docusign in the same way as the Nodestack spam was likely triggered by the mention of @HostGator.

Often, you'll see this pattern play out ad nauseam as the organisation involved takes the shotgun approach of blasting their message out over and over again:

Having watched this pattern play out over the years, the offending Twitter accounts do seem to eventually realise that the strategy is either ineffective or simply pisses too many people off and cease the spam after a little while. Nodestack haven't quite evolved to that level of understanding yet and as of today, still appear to be hijacking disgruntled threads containing keywords of "HostGator", "Bluehost" or "GoDaddy", among others. (Amusingly, they also appear to be attempting to woo happy Ghost customers - or those considering Ghost - over to WordPress.)

And just the day after first seeing Nodestack's behaviour:

The point is that Nodestack is not the sole offender here, although they are exceptionally bad in other ways. I'll come back to that.

One of the arguments I've seen bandied around in favour of this strategy is that "it's useful to people". The position here is that in a case like that last tweet by Guardian Digital, they believe that the original tweet demonstrated a need and they're simply trying to help that person out with a solution. But let's try it like this: what if every hosting company that offered a service took the same approach? I mean suspend reality for a moment and assume that AWS, Azure, Digital Ocean et al all decided this marketing strategy "enhanced consumer value" - how would that look? Well it would be unmanageable, of course, as the noise of all the spam would drown out every conversation it was interjected into. So it's one of those things where even those who promote this means of unsolicited messaging agree that it only works so long as they're the only ones doing it!

The premise of interjecting into a conversation and promoting a service to someone at a time of need goes by another name too:

So how do organisations respond once the penny drops that this practice is usually received in a very negative way? The responsible ones acknowledge the mistake and promise to do better in future:

Expecting that Nodestack would follow suite, I suggested this was a strategy worth reconsidering:

Surprisingly though, Nodestack decided to double-down and dig themselves in deeper:

I won't reproduce all the to-and-for here, but you can easily recap by browsing the thread above. Long story short, they remained defiant so I put the question out to the masses:

Again, I'll avoid reproducing all the responses here, but it's worth a spin through the thread to see Nodestack's reaction (get popcorn first - it gets weird). As for other respondents, without manually counting them all, I'd say it's somewhere in order of a 4:1 ratio of those who are staunchly against the practice (many people were very direct about those feelings), versus those who'd accept it, or at least put up with it. I appreciate that we all have different tolerances to criticism, but when the vast majority of people are saying "don't do this", you really want to think about not doing it!

So why take this approach? I mean why blast people with spam given how much it's disliked? Apparently it's a conscious target all the people strategy where you just bombard targets with enough adverts that if some bite, then you're all good (think along the lines of the economics of Nigerian scams). The fundamental misunderstanding at play in that tweet (and other similar ones) is that everyone is free to just ignore spam, so what's the problem? Of course, that's simply not how it works and indeed that's why we have laws protecting consumers from it.

Look, perhaps contrary to what the vocal opposition they received suggests, Nodestack really do pick up business by thread-hijacking, you'll have to make your own mind up on how trustworthy that statement is. But consider the truthfulness of that assertion in the context of all sorts of other oddities with the way they're doing business ranging from a fake Trustpilot rating to missing business registration data to cohabitation with spam sites to a poor SSL Labs rating to excessive open ports on the web site to a failing "F" grade on their security headers (that last one was in response to a claim their security is "water tight", a claim somehow made in ignorance of how a similar claim wound up for T-Mobile Austria only the week before). The Twitter account went uncharacteristically quiet in response to these issues, other than when demonstrating a lack of understanding as to why forward secrecy is important in this day and age. I tried to find some further info on who was behind the service, but a LinkedIn search only came up with a Rails developer in Portugal and a bloke in Bangladesh; it's not clear whether that "nodestack" is the same org nor whether those two guys are running a business from afar whilst implying it's based in the UK having only setup the website and Twitter account in Jan (although it may explain all the typos in their tweets). This is not intended purely to berate Nodestack (for all I know it's a guy in a shed just trying to make a good go of things), rather it's to illustrate the pattern of professionalism that thread-hijacking fits into.

When this blog post is later shared in the wake of subsequent thread-hijacking behaviour from other accounts, I want it to be clear that this is the sort of company spammers will be keeping. It may well work in the same way as other advertising many of us find sleazy works; if enough people pay for the service based on those ads then the end justifies the means, right? Regardless, it's hard to imagine anyone who actually values their reputation behaving in this way. Seeing this unfold last week, it immediately made me think back to my "Hack Your Career" talk from last year at NDC:

It's just a question of how valuable your reputation and character is to you.

By pure coincidence, only a couple of days ago I saw another case of thread-hijacking in response to the same original thread:

I thought I'd see how my earlier approach would go with these guys - would they push back equally as hard against common sense advice? I copied and pasted exactly the same tweet I sent to Nodestack, just for scientific purposes:

And true to their word, that was that - they cleaned up their act. It's nice to see common sense prevail.

And there’s plenty more!

I’m Pete LePage. Let’s dive in and see what’s new for developers in Chrome 66!

CSS Typed Object Model

If you’ve ever updated a CSS property via JavaScript, you’ve used the CSS object model. But it returns everything as a string.

el.style.opacity = 0.3;
console.log(typeof el.style.opacity);> 'string' // A string!?

To animate the opacity property, I’d have to cast the string to a number, then increment the value and apply my changes. Not exactly ideal.

function step(timestamp) {
  const currentOpacity = parseFloat(el.style.opacity);
  const newOpacity = currentOpacity + 0.01;
  element.style.opacity = newOpacity;
  if (newOpacity <= 1) {
    window.requestAnimationFrame(step);
  }
}

With the new CSS Typed Object Model, CSS values are exposed as typed JavaScript objects, eliminating a lot of the type manipulation, and providing a more sane way of working with CSS.

Instead of using element.style, you access styles through the.attributeStyleMap property or .styleMap. They return a map-like object that makes it easy to read or update.

el.attributeStyleMap.set('opacity', 0.3);
const oType = typeof el.attributeStyleMap.get('opacity').value;
console.log(oType);> 'number' // Yay!

Compared to the old CSS Object Model, early benchmarks show about a 30% improvement in operations per second - something that’s especially important for JavaScript animations.

el.attributeStyleMap.set('opacity', 0.3);
el.attributeStyleMap.has('opacity'); // true
el.attributeStyleMap.delete('opacity');
el.attributeStyleMap.clear(); // remove all styles

It also helps to eliminate bugs caused by forgetting to cast the value from a string to a number, and it automatically handles rounding and clamping of values. Plus, there’s some pretty neat new methods for dealing with unit conversions, arithmetic and equality.

el.style.opacity = 3;
const opacity = el.computedStyleMap().get('opacity').value;
console.log(opacity);> 1

Eric has a great post with several demos and examples in hisexplainer.

Async Clipboard API

const successful = document.execCommand('copy');

Synchronous copy & paste using document.execCommand can be OK for small bits of text, but for anything else, there’s a good chance it’s synchronous nature will block the page, causing a poor experience for the user. And the permission model between browsers is inconsistent.

The new Async Clipboard API is a replacement that works asynchronously, and integrates with the permission API to provide a better experience for users.

Text can be copied to the clipboard by calling writeText().

navigator.clipboard.writeText('Copy me!')
  .then(() => {
    console.log('Text is on the clipboard.');
  });

Since this API is asynchronous, the writeText() function returns a Promise that will be resolved or rejected depending on whether the text we passed is copied successfully.

Similarly, text can be read from the clipboard by calling getText() and waiting for the returned Promise to resolve with the text.

navigator.clipboard.getText()
  .then((text) => {
    console.log('Clipboard: ', text);
  });

Check out Jason’s post and demos in the explainer. He’s also got examples use async functions.

CSS Paint API

The canvas element lets you manipulate graphics at the pixel level, you can draw graphs, manipulate photos, or even do real time video processing. But, unless you’re starting with a blank canvas, you need a way to render an image on the canvas.

Historically, that’s meant creating an image tag, then rendering it’s contents on to the canvas. Unfortunately that means the browser needs to store multiple copies of the image in memory.

const context = el.getContext('2d');
const img = new Image();
img.onload = function () {
  context.drawImage(img, 0, 0);
}
img.src = 'llama.png';

Starting in Chrome 66, there’s a new asynchronous rendering context that’s streamlined the display of ImageBitmap objects. They now render more efficiently and with less jank by working asynchronously and avoiding memory duplication.

To use it:

1. Call createImageBitmap and hand it an image blob, to create the image.
2. Grab the bitmapprerenderer context from the canvas.
3. Then transfer the image in.

const image = await createImageBitmap(imageBlob);
const context = el.getContext('bitmaprenderer');
context.transferFromImageBitmap(image);

Done, I’ve rendered the image!

And more!

These are just a few of the changes in Chrome 66 for developers, of course, there’s plenty more.

• TextArea and Select now support the autocomplete attribute.
• Setting autocapitalize on a form element will apply to any child form fields, improving compatibility with Safari’s implementation of autocapitalize.
• trimStart() and trimEnd() are now available as the standards-based way of trimming whitespace from strings.

Be sure to check out New in Chrome DevTools, to learn what’s new in for DevTools in Chrome 66. And, if you’re interested in Progressive Web Apps, check out the new PWA Roadshow video series. Then, click the subscribe button on ourYouTube channel, and you’ll get an email notification whenever we launch a new video, or add our RSS feed to your feed reader.

I’m Pete LePage, and as soon as Chrome 67 is released, I’ll be right here to tell you -- what’s new in Chrome!

I said something about Perl 6 the other day, and someone replied asking whether anyone actually uses Perl 6. My first thought was I bet more people use Perl 6 than Haskell, and it’s well known that people use Haskell. I looked at the TIOBE Index to see whether that’s true. I won’t argue how well the index measures popularity, but for this post I’ll assume it’s a good enough proxy.

TIOBE doesn’t separate out variations on Perl [1]. What it calls Perl is 16th on the list this year, while Haskell comes in at 42nd. A few of the more obscure languages that TIOBE ranks higher than Haskell are Scratch, D, ABAP, Apex, and PL/I. Haskell has better public relations than all these languages.

There’s a lot more to viability than just popularity, though popularity matters. More users means more people to find bugs, write libraries, develop tools, answer questions, write tutorials, etc. But the benefit of community size is not linear. It goes through a sort of logistic S-curve. There’s some threshold size where the community is large enough for a language to be viable. And somewhere above that threshold you start hitting diminishing return.

It’s interesting to look at some of the languages currently less popular than Haskell but more familiar: Common Lisp (63), Erlang (66), and F# (67). These show that popularity isn’t everything.

Common Lisp has been around since 1982, and was standardizing a language that had been in development since 1958. Erlang has been around since 1986. These languages have many of the benefits of popularity listed above, accumulated over time.

There is not a huge community devoted specifically to F#, but it shares tooling and libraries with C#, the 5th language on the list. (Maybe the number of F# developers is underestimated because F# is so closely related to C#, not syntactically but in terms of infrastructure.)

Common Lisp, Erlang, and F# would all be safer bets for a production software project than several more popular languages.

***

[1] At least I don’t think they do. TIOBE does separate out some versions of Lisp as separate languages. It’s possible they do consider Perl 6 a separate language that didn’t make the top rankings.

Larry Wall deliberately introduced many natural language principles in Perl. It seems that one feature that Perl has in common with natural languages is controversy over when two dialects of a language are sufficiently different to be considered separate languages. Advocates consider Perl 6 to be a separate language but outside observers, like TIOBE, may not.

Today, Microsoft co-founder Bill Gates is one of the world's richest people, with a total net worth estimated at nearly $90 billion. But a little over four decades ago, Gates was a brash young software developer who was just getting ready to take center stage.

In fact, it was this week in 1976 that a 20-year-old Gates was preparing to make a big speech — the opening remarks at the First Annual World Altair Computer Convention in Albuquerque, New Mexico. At the time, Gates was still technically a student at Harvard University, having taken a leave of absence after two years in school to form a computer software company with his high school friend, Paul Allen. That company, of course, would become Microsoft and Gates never finished his degree (though, he was granted an honorary one in 2007).

Gates and Allen had worked together as computer programmers at software company Honeywell in the summer of 1974 following Gates' freshman year at Harvard. That same year, the world's first personal computer kit was released. The Micro Instrumentation and Telemetry Systems' (MITS) Altair 8800 inspired Gates and Allen, and they started working on software for the computer.

In a 2007 commencement speech at Harvard, Gates reminisced about calling MITS president Ed Roberts (from his dorm phone) to sell MITS on the idea of Gates building a version of the programming language BASIC for the company's new Altair computers.

"I worried that they would realize I was just a student in a dorm and hang up on me," Gates recalled. "Instead they said: 'We're not quite ready, come see us in a month,' which was a good thing, because we hadn't written the software yet.

"From that moment, I worked day and night on this little extra credit project that marked the end of my college education and the beginning of a remarkable journey with Microsoft."

In fact, Gates and Allen both admitted later that they had lied and told MITS that they already had a version of BASIC for the Altair when they hadn't yet written the software. "We had nothing," Allen told CBS' 60 Minutes in an interview.

But by the time the audacious pair met with MITS in New Mexico two months later, their programming language was ready.

After selling that software to MITS, both Gates and Allen took jobs with the company in Albuquerque at the end of 1975, while also forming their own partnership, then called "Micro-Soft." Gates and Allen continued working on developing software and, in February 1976, Gates wrote his famous "Open Letter to Hobbyists" in MITS' newsletter.

In the letter, which Gates later would say "really became a cause celebre at the time," the future billionaire complained about the widespread practice of computer users sharing the software he and Allen had developed without paying for it, calling the practice copyright infringement. Gates argued that, in order for quality software to be developed for computers, that developers like himself needed to be paid fairly. Gates included his Albuquerque address at the end of the letter for "any one who wants to pay up."

Over the ensuing decades, Gates and Microsoft would make billions of dollars precisely because of its copyrighted software is not shareable.

CARACAS (Reuters) - Chauffeured around in a sleek black pick-up, the head of Venezuela’s oil industry, Major General Manuel Quevedo, last month toured a joint venture with U.S. major Chevron.

Flanked by other trucks carrying security guards, Quevedo passed a handful of workers waiting by an oil well cluster. They wanted a word with the OPEC nation’s oil minister and president of its state-run oil firm, PDVSA [PDVSA.UL], about the sorry state of the company.

Quevedo and his caravan drove on by.

“He didn’t get out to ask workers about what is going on,” said Jesus Tabata, a union leader who works on a rig in the oil-rich Orinoco Belt. “That way it’s easier to keep saying everything is fine - and at the same time keeping us on like slaves on miserable wages.”

What’s going on is that thousands of oil workers are fleeing the state-run oil firm under the watch of its new military commander, who has quickly alienated the firm’s embattled upper echelon and its rank-and-file, according to union leaders, a half-dozen current PDVSA workers, a dozen former PDVSA workers and a half-dozen executives at foreign companies operating in Venezuela.

Some PDVSA offices now have lines outside with dozens of workers waiting to quit. In at least one administrative office in Zulia state, human resources staff quit processing out the quitters, hanging a sign, “we do not accept resignations,” an oil worker there told Reuters.

Official workforce statistics have become a closely guarded secret, but a dozen sources told Reuters that many thousands of workers had quit so far this year - an acceleration of an already troubling outflow last year.

About 25,000 workers resigned between the start of January 2017 and the end of January 2018, said union leader and government critic Ivan Freites, citing internal company data. That figure comes out of a workforce last officially reported by PDVSA at 146,000 in 2016.

Resignations appear to have increased sharply this year, said Freites, a prominent union leader at Venezuela’s major refineries in the northern Paraguana peninsula.

“It’s unstoppable,” he said.

Many of those leaving now are engineers, managers, or lawyers - high-level professionals that are almost impossible to replace amid Venezuela’s economic meltdown, the PDVSA workers and foreign executives told Reuters.

PDVSA and the Oil Ministry did not respond to repeated requests for comment. PDVSA board member and pro-government union representative Wills Rangel acknowledged the flight of talent is a serious problem.

“The massive resignations are worrying,” Rangel said in an interview. “In refinery operations, many have left.”

The pace of departures has quickened with the rapid deterioration of PDVSA’s operations and finances - radiating pain through the OPEC nation’s oil-based economy, now beset with food shortages and hyperinflation.

Quevedo - a little known former housing minister who replaced two executives jailed for alleged graft - has further poisoned the atmosphere, according to the two dozen sources who spoke with Reuters.

A stiff official who rose through the National Guard, Quevedo fired many long-term employees upon arrival and urged remaining ones to denounce any of their colleagues who oppose Maduro. He tapped soldiers for top roles, giving the oil firm the atmosphere of a “barrack,” two company sources said.

“The military guys arrive calling the engineers thieves and saboteurs,” said a Venezuelan oil executive at a private company who frequently works with PDVSA.

Quevedo is also fighting to retain control of a company increasingly riven by turf wars. The ruling socialists, once held together by late leader Hugo Chavez, have succumbed to infighting under Maduro, a former bus driver and union leader who lacks Chavez’ charisma and has seen his budget slashed with the decline in global oil prices.

Quevedo has clashed with Venezuela’s powerful Vice-President Tareck El-Aissami. When El-Aissami in February appointed a vice-president to the PDVSA unit that oversees joint ventures with foreign companies, Quevedo removed the appointee and had him arrested, according to three sources with knowledge of the incident, which has not been previously reported.

Quevedo is an ally of Socialist Party heavyweight Diosdado Cabello.

“There is a fight between Diosdado and Tareck for control of the industry,” said Hebert Garcia, a former army general who later broke with Maduro and fled the country.

The political turmoil and mass resignations threaten Maduro’s government, which depends on oil for 90 percent of export revenue.

In the Orinoco Belt, some drilling rigs are working only intermittently for lack of crews, said two sources there. In PDVSA’s refineries, several small fires have broken out because there are no longer enough supervisors, two sources in the northern Paraguana peninsula said. Lack of personnel in export terminals have forced some ports to cut back working hours, according to two shippers and one trader.

Oil production in the first quarter of this year slipped to a 33-year low of 1.6 million barrels per day.

‘WHEN ARE YOU LEAVING?’

Jobs at PDVSA were once coveted for their generous salaries and benefits, including cheap credit for housing. Now, many PDVSA workers can’t feed their families on wages that amount to a handful of U.S. dollars a month.

Rampant food shortages that caused Venezuelans to report losing an average of 11 kilograms (24 pounds) last year are particularly tough for oil workers tasked with grueling physical work in often remote oil fields.

Some oil workers have resorted to working odd jobs on the side, taking vacation to work abroad, or even selling their work uniforms - red overalls - for money to eat.

Some workers in Lake Maracaibo, a production region near Colombia, can no longer get to their jobs, according to two sources there. Transport can cost up to 55,000 bolivars - equal to only 10 U.S. cents, but close to what some workers earn in a day.

“Now what we ask each other is: ‘When are you leaving and for where?’,” said one of the Maracaibo workers, who like thousands of other Venezuelans emigrated to Colombia this month. “Even in the bathroom, people are talking about quitting.”

‘WHO WILL BE LEFT?’

At PDVSA headquarters, Quevedo often walks through the offices with a half dozen bodyguards who clear his path, according to one current and one former PDVSA employee.

The company’s ongoing decay is evident all around him in the once polished office tower: Broken elevators, poor cafeteria food, empty desks in once-crowded divisions.

Maduro has overseen the arrest of dozens of high-level PDVSA executives since late last year, sometimes at the Caracas headquarters as shocked employees looked on. Workers now feel watched by supervisors and are loathe to make any business decision out of fear they will later be accused of corruption, the sources said.

PDVSA workers, often visibly thinner, sometimes surreptitiously hand out resumes to executives from private companies, according to a source at a foreign firm.

In a rare protest last month, angry Oil Ministry workers blocked access to the cafeteria, demanding better benefits and chanting that Quevedo should resign.

Venezuela’s foreign oil partners, which include California-based Chevron, Russia’s Rosneft and China’s CNPC, are increasingly worried about PDVSA’s rapidly departing workforce, according to a half-dozen sources at multinational companies operating in Venezuela. But as minority partners, they have little or no sway over salaries and management.

The foreign partners have also grown increasingly frustrated with Quevedo, who initially asked for their suggestions on fixing the state-run firm but now appears ill-disposed toward reforms, the sources said.

At least one foreign company is considering bringing in foreign specialists to improve its operations, one of the sources added. But with crime, power cuts and shortages rampant in Venezuela, luring foreign professionals is tough.

Still, in the Orinoco belt, some vow to stay in the belief that Maduro’s government can’t last.

“We can’t give up,” said Tabata, the union leader who watched Quevedo’s truck drive by that day. “This government is unstable and could fall at any moment - and who will be left?”

Offentliggjort den 17. mar. 2014

By Jonathan Corbet
February 19, 2018

It may be tempting to think that iptables has been the kernel's packet-filtering implementation forever, but it is a relative newcomer, having been introduced in the 2.4.0 kernel in 2001. Its predecessors (ipchains, introduced in 2.2.10, and ipfwadm, which dates back to 1.2.1 in 1995) are mostly forgotten at this point. Iptables has served the Linux community well and remains the firewalling mechanism that is most widely used, but it does have some shortcomings; it has lasted longer than the implementations that came before, but it is clearly not the best possible solution to the problem.

The newer nftables subsystem, merged for the 3.13 kernel release in early 2014, introduced an in-kernel virtual machine to implement firewall rules; users have been slowly migrating over, but the process has been slow. For some strange reason, system administrators have proved reluctant to throw away their existing firewall configurations, which were painful to develop and which still function as well as they ever did, and start over with a new and different system.

Still, it was logical to assume that nftables would eventually take over, especially as the iptables compatibility layers improved. Some people started to doubt this story, though, when serious development started on the BPF virtual machine. There seemed to be a lot of overlap between the two virtual machines, and BPF was being quickly extended in ways that improved its performance, functionality, and security. Even so, nftables development has continued, and there has been little talk — until now — of pushing BPF into the core of the firewalling code.

Bringing in BPF

The announcement of bpfilter changes that situation, though. In short, bpfilter enables the creation of BPF programs that can be attached to points in the network packet path and make filtering decisions. In the proof-of-concept patches, those programs are attached at the express data path (XDP) layer, where they are run from the network-interface drivers. But, as Daniel Borkmann noted in the introduction to the patches, BPF programs could be just as easily attached at any other point in the path, allowing them to make decisions at the same points that iptables rules do.

There are a number of advantages claimed for the bpfilter approach. BPF programs can be just-in-time compiled on most popular architectures, so they should be quite fast. The work that has been done to enable the offloading of XDP-level programs to the network interface itself can come into play here, moving firewall processing off the host CPU entirely. The use of BPF enables the writing of firewall rules in C, which may appeal to some developers who are starting from the beginning. And firewall code would be subject to the BPF verifier, adding a layer of security to the whole system.

One of the core design features for bpfilter is the ability to translate existing iptables rules into BPF programs. This feature is intended to make it easy for existing firewall configurations to be moved over to the new scheme, perhaps without system administrators even knowing that it is happening. This translation is done in an interesting manner. Iptables rules are passed to the kernel, so the kernel must take responsibility for doing that work, but the task can be a complex one that would benefit from a user-space implementation.

To enable such an implementation, the bpfilter developers have created a new mechanism that supports the creation of a special type of kernel module to handle this kind of task. These modules would be part of the kernel and would be shipped by distributors as just another .ko file, but they would contain an ordinary ELF executable. After the module has been loaded, its code can be run in a separate user-space process; all that is required is a call to a special version ofcall_usermodehelper().

This mechanism allows the translation code to be managed as if it were just another part of the kernel. That code can be developed in user space, though. When it runs, the translation code will be separated from the kernel, making it harder to attack the kernel via that path. If this mechanism catches on, one can imagine that a number of other tasks could eventually be pushed out of the kernel proper into one of these special user-space modules. Developers should be careful, though; this could prove to be a slippery slope leading toward something that starts to look like a microkernel architecture.

Early responses

There have not been a whole lot of comments thus far on the code itself. That may be partly because, in their haste to get a proof of concept out to illustrate the idea, the developers never quite got around to writing comments in the code — or even changelogs for the patches. The idea itself, though, has raised concerns for some developers.

Harald Welte, who is not often seen in this community these days, showed up with a number of questions. At the top of his list was the decision to emulate iptables rules with the new BPF mechanism. If the new subsystem is to ever replace the iptables implementation, it will need to implement exactly the same behavior; small and subtle differences could introduce security problems into deployed firewall configurations. Given the complexity of iptables, the chances of such differences happening are significant.

More fundamentally, the networking developers have wanted to phase out iptables and its user-space interfaces for some time. Iptables has not aged entirely well. For example, there is no way to add or replace a single rule (or small set of rules); iptables can only wipe out the entire configuration and start from scratch. That makes firewall changes expensive; it also gets difficult to coordinate changes when they are being made by multiple actors at once. The increasing use of containers has created just this kind of situation; addressing this problem requires moving away from the iptables API. The fact that iptables requires separate rule sets for IPv4 and IPv6 creates a pain point for administrators as well.

Implementing the iptables API with bpfilter, Welte said, will "risk perpetuating the design mistakes we made in iptables some 18 years ago for another decade or more". It will push back the (already distant) date when that API could be deprecated and removed. Rather than focusing on iptables, Welte said, the developers should create an emulation of the newer nftables API, which was designed with the lessons from iptables in mind. That would support sites that have already migrated and encourage that migration to continue.

Networking maintainer David Miller (who authored some of the new code) replied that iptables is still far more widely used, so implementing that interface provides for better testing coverage in the near term. Welte answered, though, that most of the biggest use cases (Docker and Kubernetes, for example) use the command-line tools rather than the iptables API, so there is no need to implement emulation of the API itself to test with those systems. Miller, however, disagreed with the idea that the iptables binaries could be easily replaced on deployed systems: "Like it or not iptables ABI based filtering is going to be in the data path for many years if not a decade or more to come".

Interestingly, while there was talk of implementing the nftables API, nobody has yet questioned the idea of applying the BPF virtual machine to firewalls, even though it would be likely to supplant nftables relatively quickly. Instead, Miller said in the discussion that nftables failed to address the performance problems in Linux's packet-filtering implementation, driving users toward user-space networking technologies instead. There is a real possibility that nftables could end up being one of those experiments that is able to shed some light on the problem space but never takes over in the real world.

Overall, bpfilter is an extremely young project and there are a lot of questions yet to be answered about it. While much of the packet-filtering logic can likely be expressed in BPF code, there are more advanced features (like connection tracking, pointed out by Florian Westphal) that are still likely to need a fair amount of kernel support. There are no performance numbers with the patch set, so any performance gains are still theoretical at this point. And the code itself is quite young, lacking both features and documentation.

The end result is that we'll probably not see bpfilter in the mainline kernel in the immediate future. Given the developers who have worked on it, though, bpfilter is clearly a serious initiative that is firmly aimed at getting into the mainline eventually. If it truly proves to be a better solution to the network packet-filtering problem, those developers seem likely to prevail eventually.

Ecosystem

ClojureScript and Clojure

ClojureScript and npm

Immutable Data Structure

Communities

Mux is building the future of online video infrastructure.

Our first product was analytics and performance monitoring - think “NewRelic for Video” - used by major video streaming companies like PBS, Vimeo, and the New York Times to monitor and improve billions of streams.

In early 2018, we launched Mux Video, a simple API to video hosting, encoding and streaming - think "Stripe for Video" - which uses our performance data, machine learning, and just-in-time encoding for unmatched video quality and efficiency. POST a video in; GET a video out that plays anywhere and is deeply optimized for the target device and user. Then get creative with features like audio transcription and face detection. We think building with video should be fun, not complicated.

You’ll be joining an amazing team from places like Google/YouTube, Facebook, Twitch, Brightcove, and Bain. Our founders previously started (and sold) Zencoder, an early leader in cloud video technology. We also authored Video.js, the biggest HTML5 video player on the web. We were named one of the best Y Combinator startups of 2016, and we recently raised a Series A from Accel, joining existing investors SV Angel, Lowercase, the founders of Heroku, Parse, and others.

The New Hyper 2.0 and the Hyper Store

• A new high-performance Canvas based rendering engine
• An online catalog of plugins and themes
• The hyper CLI to install plugins and themes with one command
• Improved keymaps support
• Improved support for hyperlinks

Can Hyper now seamlessly handle lots of streaming output? `yes`!

Using the `hyper-broadcast` plugin, we synchronize `parrot.live`🔴

The Hyper Store gathers the best plugins and themes from npm

hyper i hyperpower

An example of installing the `hyperpower` plugin

{
  'tab:new': ['command+t'],
  'tab:jump:prefix': ['command]'],
  ...
}

The improved `keymaps` object

• Make Hyper the most performant and versatile cross-platform terminal out there. We are excited to begin experimenting with WebGL and WebAssembly to push the boundaries of web-based applications.

• Provide the necessary hooks to make the most creative terminal extensions. As we've shared before, we believe that command-based UI is uniquely productive, can be user-friendly and has an incredibly bright future ahead.

And there’s plenty more!

I’m Pete LePage. Let’s dive in and see what’s new for developers in Chrome 66!

CSS Typed Object Model

If you’ve ever updated a CSS property via JavaScript, you’ve used the CSS object model. But it returns everything as a string.

el.style.opacity = 0.3;
console.log(typeof el.style.opacity);> 'string' // A string!?

To animate the opacity property, I’d have to cast the string to a number, then increment the value and apply my changes. Not exactly ideal.

function step(timestamp) {
  const currentOpacity = parseFloat(el.style.opacity);
  const newOpacity = currentOpacity + 0.01;
  element.style.opacity = newOpacity;
  if (newOpacity <= 1) {
    window.requestAnimationFrame(step);
  }
}

With the new CSS Typed Object Model, CSS values are exposed as typed JavaScript objects, eliminating a lot of the type manipulation, and providing a more sane way of working with CSS.

Instead of using element.style, you access styles through the.attributeStyleMap property or .styleMap. They return a map-like object that makes it easy to read or update.

el.attributeStyleMap.set('opacity', 0.3);
const oType = typeof el.attributeStyleMap.get('opacity').value;
console.log(oType);> 'number' // Yay!

Compared to the old CSS Object Model, early benchmarks show about a 30% improvement in operations per second - something that’s especially important for JavaScript animations.

el.attributeStyleMap.set('opacity', 0.3);
el.attributeStyleMap.has('opacity'); // true
el.attributeStyleMap.delete('opacity');
el.attributeStyleMap.clear(); // remove all styles

It also helps to eliminate bugs caused by forgetting to cast the value from a string to a number, and it automatically handles rounding and clamping of values. Plus, there’s some pretty neat new methods for dealing with unit conversions, arithmetic and equality.

el.style.opacity = 3;
const opacity = el.computedStyleMap().get('opacity').value;
console.log(opacity);> 1

Eric has a great post with several demos and examples in hisexplainer.

Async Clipboard API

const successful = document.execCommand('copy');

Synchronous copy & paste using document.execCommand can be OK for small bits of text, but for anything else, there’s a good chance it’s synchronous nature will block the page, causing a poor experience for the user. And the permission model between browsers is inconsistent.

The new Async Clipboard API is a replacement that works asynchronously, and integrates with the permission API to provide a better experience for users.

Text can be copied to the clipboard by calling writeText().

navigator.clipboard.writeText('Copy me!')
  .then(() => {
    console.log('Text is on the clipboard.');
  });

Since this API is asynchronous, the writeText() function returns a Promise that will be resolved or rejected depending on whether the text we passed is copied successfully.

Similarly, text can be read from the clipboard by calling getText() and waiting for the returned Promise to resolve with the text.

navigator.clipboard.getText()
  .then((text) => {
    console.log('Clipboard: ', text);
  });

Check out Jason’s post and demos in the explainer. He’s also got examples use async functions.

CSS Paint API

The canvas element lets you manipulate graphics at the pixel level, you can draw graphs, manipulate photos, or even do real time video processing. But, unless you’re starting with a blank canvas, you need a way to render an image on the canvas.

Historically, that’s meant creating an image tag, then rendering it’s contents on to the canvas. Unfortunately that means the browser needs to store multiple copies of the image in memory.

const context = el.getContext('2d');
const img = new Image();
img.onload = function () {
  context.drawImage(img, 0, 0);
}
img.src = 'llama.png';

Starting in Chrome 66, there’s a new asynchronous rendering context that’s streamlined the display of ImageBitmap objects. They now render more efficiently and with less jank by working asynchronously and avoiding memory duplication.

To use it:

1. Call createImageBitmap and hand it an image blob, to create the image.
2. Grab the bitmapprerenderer context from the canvas.
3. Then transfer the image in.

const image = await createImageBitmap(imageBlob);
const context = el.getContext('bitmaprenderer');
context.transferFromImageBitmap(image);

Done, I’ve rendered the image!

And more!

These are just a few of the changes in Chrome 66 for developers, of course, there’s plenty more.

• TextArea and Select now support the autocomplete attribute.
• Setting autocapitalize on a form element will apply to any child form fields, improving compatibility with Safari’s implementation of autocapitalize.
• trimStart() and trimEnd() are now available as the standards-based way of trimming whitespace from strings.

Be sure to check out New in Chrome DevTools, to learn what’s new in for DevTools in Chrome 66. And, if you’re interested in Progressive Web Apps, check out the new PWA Roadshow video series. Then, click the subscribe button on ourYouTube channel, and you’ll get an email notification whenever we launch a new video, or add our RSS feed to your feed reader.

I’m Pete LePage, and as soon as Chrome 67 is released, I’ll be right here to tell you -- what’s new in Chrome!

Did you ever feel like you have too much storage?

Probably not - there is no such thing as ‘too much storage’. For a long time, we have used userland tools like gzip and rar for compression. Now with Virtual Data Optimizer (VDO), all required pieces for a transparent compression/deduplication layer are available in the just-released Red Hat Enterprise Linux 7.5. With this technology, it is possible to trade CPU/RAM resources for disk space. VDO becoming available is one of the results of Red Hat acquiring Permabit Technology Corporation in 2017. The code is available in the source RPMs, and upstream projects are getting established.

Regarding use cases, VDO can, for example, be used under local filesystems, iSCSI or Ceph. File servers can use it as base for local filesystems, and hand out NFS, CIFS or Gluster services. Remember dozens of Linux systems sharing read-only NFS root file systems to save storage? You could now give each of these systems its own individual image via iSCSI, stored on a common file system with VDO backend, and have VDO deduplicate and compress the common parts of these images.

When reading about VDO appearing in the Red Hat Enterprise Linux 7.5 beta, I wondered about the following:

• How to setup and configure VDO?

• How is VDO influencing read/write performance?

• How much storage space can I save for my use cases?

Let’s find out!

How to setup and configure VDO?

The authoritative resource regarding VDO is the Storage Administration Guide. With Red Hat Enterprise Linux 7.5 repos available, it can be installed using:

[root@rhel7u5a ~]# yum install vdo kmod-kvdo

When configuring VDO, various things have to be taken into consideration. The smallest system I used here is a KVM guest with 2GB of RAM. While sufficient for first tests, for production one should obey the sizing recommendations in the Storage Administration Guide. They depend on the size of the storage below your VDO devices. Attention should also be paid to the order of layers: placing VDO below encryption, for example, makes no sense.

Assuming a disk available as /dev/sdc, we can use the following command to create a VDO device on top. For a 10GB disk, depending on the workload, one could decide to have VDO offer 100GB to the upper layers:

[root@rhel7u5a ~]# vdo create --name=vdoasync --device=/dev/sdc \   --vdoLogicalSize=100G --writePolicy=asyncCreating VDO vdoasyncStarting VDO vdoasyncStarting compression on VDO vdoasyncVDO instance 0 volume is ready at /dev/mapper/vdoasync[root@rhel7u5a ~]#

After creating a filesystem using ‘mkfs.xfs -K /dev/mapper/vdoasync’, the device can be mounted to /mnt/vdoasync.

VDO supports three write modes.

• The ‘sync’ mode, where writes to the VDO device are acknowledged when the underlying storage has written the data permanently.

• The ‘async’ mode, where writes are acknowledged before being written to persistent storage. In this mode, VDO is also obeying flush requests from the layers above. So even in async mode it can safely deal with your data - equivalent to other devices with volatile write back caches. This is the right mode, if your storage itself is reporting writes as ‘done’ when they are not guaranteed to be written.

• The ‘auto’ mode, now the default, which selects async or sync write policy based on the capabilities of the underlying storage.

Sync mode commits the data out to media before trying to either identify duplicates or pack the compressed version with other compressed blocks on media. This means sync mode will always be slower on sequential writes, even if reads are much faster than writes. However, sync mode introduces less latency, so random IO situations can be much faster. Sync mode should never be used in production without backed storage (typically using batteries or capacitors) designed for this use case.

Using ‘auto’ to select the mode is recommended for users. The Storage Admin Guide has further details regarding the write modes.

How is VDO influencing read/write performance?

So if we put the VDO layer between a file system and a block device, how will it influence the I/O performance for the filesystem?

The following data was collected using spinning disks as backend, on a system with 32 GB RAM and 12 cores at 2.4Ghz.

Column ‘deploy to file system’ shows the time it took to copy a directory with ~5GB of data from RAM (/dev/shm) to the filesystem. Column ‘copy on file system’ shows the time required to make a single copy of the 5GB directory which was deployed in the first step. The time was averaged over multiple runs, always after removing the data and emptying the file system caches. Publicly available texts from the Project Gutenberg were used as data, quite nicely compressible.

Deployment to VDO backend is slower than to plain LVM backend.

I was initially wondering whether copies on top of a VDO backed volume would be faster than copies on top of a LVM volume - they are not. When data on a VDO backed file system is copied, for example with ‘cp’, then the copy is by definition a duplicate. After VDO identifies this as candidate for duplicate data, it does a read comparison to be sure.

When using SSD or NVDIMM as backend, extra tuning should be done. VDO is designed to properly deal with many parallel I/O requests in mind. In my tests here I did not optimize for parallelization, I just used single instances of ‘rsync’, ‘tar’ or ‘cp’. Also for these, VDO can break up requests from applications to write big files into many small requests - if the underlying media is, for example, a high-speed NVMe SSD, then this can help performance.

Something good to know for testing: I noticed GNU tar reading data incredibly fast when writing to /dev/null. Turns out that ‘tar’ is detecting this situation and not reading at all. So ‘tar cf /dev/null /usr’ is not doing what you probably expect, but ‘tar cf - /usr|cat >/dev/null’ is.

How much storage space can I save for my use cases?

This depends of course on how compressible your data is - creating a test setup and storing your data directly on VDO is a good way to find out. You can also to some degree decide how many cpu/memory resources you want to invest and tune for your use case: deduplication and compression can be enabled separately for VDO volumes.

VDO reserves 3-4GB of space for itself: using a 30GB block device as the VDO backend, you can use around 26GB to store your real data, so not what, for example, the filesystem above stores, but what VDO after deduplication/compression needs to store. The high overhead here is the result of the relatively small size of the device. On larger (multi-TB) devices, VDO is designed to not incur more than 2% overhead.

VDO devices can, and should use thin provisioning: this way the system reports more available space to applications than the backend actually has. Doing this, you can benefit from compression and deduplication. Like LVM volumes, VDO devices can after initial creation be grown on-the-fly.

The best data for monitoring the actual fillstate is from ‘vdostats --verbose’, for example for a 30GB volume:

[root@rhel7u5a ~]# vdostats --verbose /dev/mapper/vdoasync |grep -B6 'saving percent' physical blocks                  : 7864320 logical blocks                   : 78643200 1K-blocks                        : 31457280 1K-blocks used                   : 17023768 1K-blocks available              : 14433512 used percent                     : 54 saving percent                   : 5

So this volume is 54% full. This is a thin provisioned volume with 30GB backend, so 7864320 blocks of 4k size. We show ten times of that to the upper layers, so 300GB, here appearing as ‘logical blocks’.

As we are dealing with compression/deduplication here, having a 30GB backend does not mean that you can only store 30GB: if your data can be nicely deduplicated/compressed, you can store much more. Above output of ‘vdostats’ is from a volume with 13GB of data. After making a copy of that data on the volume, ‘df’ shows 26GB of data on the filesystem. Looking at ‘vdostats’ again, we nicely see dedup in action:

[root@rhel7u5a ~]# vdostats --verbose /dev/mapper/vdoasync |grep -B6 'saving percent' physical blocks                  : 7864320 logical blocks                   : 78643200 1K-blocks                        : 31457280 1K-blocks used                   : 17140524 1K-blocks available              : 14316756 used percent                     : 54 saving percent                   : 52

Thanks to dedup, the copy of these 13GB occupies just ~120MB on the VDO layer!

Let’s copy data to VDO backed devices (separating sync and async mode) and to devices with a plain block device backend. After an initial copy of a directory with 5GB data to the mountpoint, three copies of that directory on the device were created.

From the earlier tests we know that the initial deployment to the plain backend finishes first, this can also be seen here. The violet and green lines illustrate the blocks used by our data on VDO devices. Async and sync mode are similar in this aspect. Both VDO volumes also start with reporting ‘0 bytes occupied’ via the ‘df’ command, but we see that right from the start some blocks are used internally. For the VDO backends, we see that the initial copy takes ~50 seconds, then the copies on top of VDO start. Due to deduplication, almost no additional blocks are taken into use in that phase, but we see how ‘used bytes’, so what the filesystem layer reports, is growing.

This graph can be different, depending on the data which is copied, RAM, cpu performance and the backend which is used.

Final thoughts

When designing VDO, to date the focus was to serve as a primary storage service. It is is especially designed for high performance in environments with random IO, so using VDO as shared storage with multiple tasks on top doing I/O. Especially use cases like running multiple VMs on a single VDO volume let VDO shine.

We have seen that VDO is quite easy to use. I have not done any tuning here - but learned many things around benchmarking. ‘man vdo’ has details about many tuning options like read caches and so on. The Block Deduplication and Compression with VDO video from Devconf 2018 is good, more details are also in the article Understanding the concepts behind Virtual Data Optimizer (VDO) in RHEL 7.5 Beta from Nikhil Chawla.

Christian Horn is a Red Hat TAM based in Tokyo, working with partners and customers. In his work as Linux Engineer/Architect in Germany since 2001, later as Red Hat TAM in Munich and Tokyo, Virtualization, operations and performance tuning are among the returning topics of this daily work. In his spare time, Christian works on improving his Japanese, cycles to Tokyo’s green spots and enjoys onsen, Japanese bathes. More posts from Christian are here.

A Red Hat Technical Account Manager (TAM) is a  specialized product expert who works collaboratively with IT organizations to strategically plan for successful deployments and help realize optimal performance and growth. The TAM is part of Red Hat’s world classCustomer Experience and Engagement organization and provides proactive advice and guidance to help you identify and address potential problems before they occur. Should a problem arise, your TAM will own the issue and engage the best resources to resolve it as quickly as possible with minimal disruption to your business.

Connect with TAMs at a Red Hat Convergence event near you! Red Hat Convergence is a free, invitation-only event offering technical users an opportunity to deepen their Red Hat product knowledge and discover new ways to apply open source technology to meet their business goals. These events travel to cities around the world to provide you with a convenient, local one-day experience to learn and connect with Red Hat experts and industry peers.

Open source is collaborative curiosity. Join us at Red Hat Summit, May 8-10, in San Francisco to connect with TAMs and other Red Hat experts in person! Register now for only US$1,100 using code CEE18.

Warnings about looming public pension disasters have regularly cropped up since the 1950s, pointing to problems 25 years or more down the line. To politicians and union leaders, the troubles were someone else's predicament. Then crisis fatigue set in as the big problem remained down the road.

Today, the hard stop is five to 10 years away, within the career plans of current officials.  In the next decade, and probably within five years, some large states are going to face insolvency due to pensions, absent major changes.

There are some reassuring facts. Many states are in pretty good shape, and many others still have time and resources to fix things.  There is no serious chance of retirees being impoverished. What's in doubt is whether states will pay promised benefits to retirees with large pensions or significant outside income or assets.  Also, although most of the problem is created by politicians and union leaders cutting deals to promise future unfunded benefits to keep voters  happy, there are also plenty of stories of politicians and union leaders risking their careers to stand up for honest pensions.

It's important to distinguish between actuarial problems (the present value of projected future benefit payments exceeds the funds set aside to pay them plus projected future contributions) and cash problems (not having the money to send out this month's checks). Actuarial problems are always debatable and usually involve the distant future. Cash shortfalls are undeniable and immediate.

New Jersey has $78 billion in its state pension fund, which is supposed to cover future payments with a present value of $280 billion. But that latter number is a projection.  You can ignore it if you wish,  or hope that soaring investment returns or a pandemic among retired workers will fix it. A more certain figure is that the $78 billion represents less than seven years of required cash payments.

If we extrapolate from the past, rather than use promises in the state budget, current employees plus the state will contribute about $25 billion over those seven years, which could provide another few years before the till is empty. But it will also add around $60 billion of future liabilities to current employees. The system probably breaks down before the pension fund gets to zero, for example if assets were to fall below $30 billion while projected future liabilities exceeded $300 billion. Even the most optimistic people would have to admit the situation is unsustainable. This could happen in three years in a bad stock market, or perhaps 10 with good stock returns. But fund assets are so low relative to payouts that good returns aren't that helpful.

The next phase of public pension reform will likely be touched off by a stock market decline that creates the real possibility of at least one state fund running out of cash within a couple of years. The math says that tax increases and spending cuts cannot do much. For one thing, as we learned from Detroit, at a certain point high taxes and poor services force people and businesses out. The numbers are just too big in some states to come out of the budgets. For another, voters won't stand for it. The voters in these states have refused for decades to pay the full costs of the services they were already enjoying; they're not going to have sudden conversions to paying full costs, plus the accumulated costs from the past. State constitutions will be amended if necessary and big legal battles will be fought. I cannot see any plausible scenario in which full promised benefits are paid.

I hope that the problems of the least responsible states will shock the rest of the country into more rational reforms. Actuarial problems 25 years in the future can be solved with only moderate pain today. Cash flow problems three years in the future require chainsaws, not pens. But history does not inspire confidence that warnings will be headed.

To contact the author of this story:
Aaron Brown at aaron.brown@privateeram.com

To contact the editor responsible for this story:
Max Berley at mberley@bloomberg.net

Promises are to this point a very popular concept in JavaScript – they are native across all modern browsers, Node is switching to promises from callback style (there is even a way to convert callback style to promise style), but there are some small things which are easy to overlook.

Promisifying sync values

Sometimes we receive a value, but we are not sure whether it is promise or not. In this case, we can use Promise.resolve, which wraps the value into a promise, but if the value itself is a promise, it will wait until its resolution, so it is totally safe:

Promise.resolve(syncValue).then(sameSyncValue=>...);Promise.resolve(promiseValue).then(resolvedPromiseValue=>...);

If the value is sync, handler will be executed as a microtask (read more in Jake Archibald’s article), so it should be executed right after current sync expressions, but before next tick.

This technique is also handy in case you have a function which might return sync values, but in some cases has to execute some async code – you can just switch to promise interface, wrapping sync values in Promise.resolve().

if we use async functions and don’t use any await inside, and just return sync value, it works like it was wrapped in Promise.resolve()

Changing data from promise

Sometimes we want to change data coming from an async action – e.g. we want to add one more object before returning the promise. We can do that easily using the fact that in the promise chain functions we can safely return sync values, so we need just to write a custom then handler:

returngetUserSubscriptions(user).then(subscriptions=>{// change resolved value – add user object as wellreturn{subscriptions,user};});

So, instead of return original promise, we handle success callback by modifying resolved promise value, and return it from our function. Also, you can use this technique just to modify data from your promise (without adding new parts) – for example, if you use adapter pattern to match data which your application expects:

// we modify data from the server to serve our application better:returngetUserData().then(parseUserData);

there is also a related article about passing data between promise callbacks from Dr. Axel Rauschmayer

Promise.race caveat

Promise.race allows us to create a promise, which will be resolved (or rejected) as soon as only one promise from the array is resolved (or rejected).

There is one caveat, which totally makes sense, but easy to overlook: it we pass an empty array into Promise.race([]), it will never be resolved. This is intended, and specificed in the specification (see Note 1), so if you have a dynamically constructed array for Promise.race, wrap it in a length check condition:

if(promises.length===0){returnPromise.resolve();}else{returnPromise.race(promises);}

Promises with timeout

One possible usage of Promise.race() is to set a timeout for our promise – so, let’s say we want to tell user that everything is good anyway after 5 seconds. In order to do so, we’ll need to wait for the first resolved promise – it is not that important for us, what will be resolved first, but we definitely don’t want to wait 5 seconds all the time:

// simple implementation of `wait`, `delay`, `sleep` etc// function which resolves after passed time in msfunctionwait(ms){returnnewPromise(resolve=>setTimeout(resolve,ms));}// our logic:constpromise=makeAsyncAction();// our promise will be resolved either after our async action// or, if it takes more than 5 seconds, after these 5 secondsconstlimitedFor5SecondsPromise=Promise.race([promise,wait(5000)]);// you might want to return both promises, in case async one fails// or just to show background loaderreturn{promise,limitedFor5SecondsPromise};

As I mentioned in the commentary before returning, you still might want to have original promise reference: you can handle a failure, and show some background loader indicating that something is still in the progress, but it does not block the UI.

2nd argument in then

Promise.prototype.then accepts two parameters – first one is a success handler, and second one is for a failure. While it resembles catch a lot, they are in fact different.

Let’s put an example first:

// first waysomeAsyncCode().then(successHandler).catch(errorHandler);// another approachsomeAsyncCode().then(successHandler,errorHandler);

First things first – catch is nothing else but then(undefined, errorHandler). With this in mind, we can rewrite our first example as:

someAsyncCode().then(successHandler).then(undefined,errorHandler);

So, what is the difference? The difference is in what exactly is handled by our errorHandler– it handles previous promise in the chain: in the first case, it is someAsyncCode().then(successHandler), and in the second one, it is only someAsyncCode(). So, first case handles both original call and successHandler, but the second one, in case of throwing or returning rejected promise in successHandler, won’t handle it.

This is not what you usually expect – normally you just want to handle all promises before previous catch handler, so to avoid confusion Bluebird recommends to avoid it.

Execution order

This is not a caveat really, but promises make it very easy to chain async actions, even if they can be done in parallel:

returngetUser(userId).then(user=>{returngetSubscriptions(userId).then(subscriptions=>({subscriptions,user}))}).then(data=>{returngetFavourites(userId).then(favourites=>({...data,favourites}));});

In this example we had to pass data around, so it is not as elegant, but imagine, we want to make several POST requests, and don’t want necessarily use result from them:

returnupdateUser(user).then(()=>updateMarketing(marketing)).then(()=>updateSubscriptions(subscriptions));

In this example we don’t depend on results from the previous call, and single thing which is important for us is when all calls were finished (or if any failed). Moreover, it has problems – in case any call fails, following functions won’t even execute – even though we might be totally fine updating subscriptions in case marketing updates failed.

You can use Promise.all to make all these requests in parallel (it will return a new promise, which will be resolved as soon as all promises are resolved, with an array of resolved values):

// first examplereturnPromise.all([getUser(userId),getSubscriptions(userId),getFavourites(userId)]).then(([user,subscriptions,favourites])=>({user,subscriptions,favourites}));// second examplereturnPromise.all([updateUser(user),updateMarketing(marketing),updateSubscriptions(subscriptions)]);

Now all requests will be done in parallel, bringing you speed. It also shows that these requests don’t depend on each other, and won’t create confusion in the future, in case of refactoring. These are fairly simple examples, but it is totally fine to have several parallel promise “tracks”, which will be merged in the end – in case of big number of requests and so fast network, it might improve response time significantly.

Bitcoin is at an inflection point.

Even with the price back above $8,000, many of the so-called miners that perform the complex calculations to generate the digital currency are at risk of turning unprofitable. If prices drop below that threshold again for an extended period, there will likely be a swifter consolidation to industrial-scale mining. That could knock out the last guy-in-a-dorm-room operations and leave holders of the currency vulnerable to the dictates of the big miners.

“It’s totally different this year than last year,” Silicon Valley venture capitalist Bill Tai said in an interview. “The bitcoin mining industry was this mysterious dark cottage industry, and it’s about to grow up and about to have elements of institutional scalability at all levels."

Smaller miners will drop out, and only five to 10 of the largest will survive and be profitable, said Tai, who serves as chairman of Hut 8 Mining Corp., the capital financing arm in North America for Amsterdam-based Bitfury Group Ltd., one of the biggest makers of crypto-mining equipment. When the price of Bitcoin approached $19,000 last year, Bitfury had to turn some customers away as $1.9 billion of orders streamed in, he said.

More concentration could also hold a lot of sway over Bitcoin’s price. Miners hold between 20 percent and 30 percent of all Bitcoins, according to Lucas Nuzzi, a senior analyst at Digital Asset Research. Bitfury alone has mined more than 1 million Bitcoins, Tai said. The company has already sold coins to defray operating expenses. If miners are forced to sell more, that could depress Bitcoin’s price.

With mining power -- so-called hashrate -- aggregated in fewer hands, that also increases the risk that a few miners could band together to execute a so-called 51-percent attack -- where they would control enough of the transactions to dictate changes in the development of the blockchain to suit their preferences.

QuickTake: All About Bitcoin, Blockchain and the Crypto World

"It has the potential of being dangerous from a security standpoint, since a single entity could use its power in terms of hashrate to disrupt the network," Nuzzi said.

The largest miners are getting bigger. Industry leader Bitmain, which runs China’s two largest bitcoin-mining collectives, has opened offices in Switzerland and the U.S. and is exploring expansion opportunities worldwide. Bitfury will be raising more than $100 million in the next six months to produce and sell portable mining server farms, each costing at least $2 million, Tai said. It’s also opening up its own, additional farms in the Middle East and Africa, and expanding in Canada. Bitfury is profitable at current prices largely due to its economies of scale, Tai said.

Biggest Expense

Unlike smaller miners, which buy hardware from other manufacturers, Bitmain and Bitfury design and make their own chips and machines. The large players also can place huge orders for parts and even buy electricity at a discount.

“We can buy and source orders of magnitude more," Tai said. "We can buy silicon in large quantities and commit to the electricity grid in chunk sizes. We have the cash to make the deposits and set them up." The company is in discussions to connect to a nuclear power plant that’s going online in about 18 months, he said.

Many smaller miners are trying to cut costs by relocating to locations such as Springfield, Missouri, where buildings -- abandoned factories and car dealerships -- can be had for as little as $1 a square foot. Electricity can cost between $3,224 and more than $9,000 to mine a single Bitcoin, depending on the state, according to Crescent Electric Supply Company’s estimates.

Power-Hungry Crypto Mines Clean Up as Cost of Electricity Grows

At current prices, though, all that may not help the smaller Bitcoin miners that much. Take startup Bcause, which hosts some 11,000 mining machines belonging to clients in Virginia Beach, Virginia. Its clients likely only make a return on their capital investments with Bitcoin above $9,000 to $10,000, founder Tom Flack said.

For now, Flack and his customers are betting that Bitcoin’s price will rise. Having only opened its facility in February, Bcause is already overbooked until September and is building another facility, as speculators continue to count on higher prices, he said.

But even Flack expects to see miner consolidation in the next 18 to 24 months.

"We’ve returned to some level of rationality," Flack said.