Media playback is unsupported on your device

YouTube stars are being paid to sell academic cheating, a BBC investigation has found.

More than 250 channels are promoting EduBirdie, based in Ukraine, which allows students to buy essays, rather than doing the work themselves.

YouTube said it would help creators understand they cannot promote dishonest behaviour.

Sam Gyimah, Universities Minister for England, says YouTube has a moral responsibility to act.

He said he was shocked by the nature and scale of the videos uncovered by the BBC: "It's clearly wrong because it is enabling and normalising cheating potentially on an industrial scale."

The BBC Trending investigation uncovered more than 1,400 videos with a total of more than 700 million views containing EduBirdie adverts selling cheating to students and school pupils.

EduBirdie is based in Ukraine, but aims its services at pupils and students across the globe.

Essay writing services are not illegal, but if students submit work they have paid for someone else to do the penalties can be severe.

The company is not just aiming to capture the attention of university students with its advertising.

Popular YouTubers, some as young as 12, are being paid to personally endorse the service.

In some of the videos YouTubers say if you cannot be bothered to do the work, EduBirdie has a "super smart nerd" who will do it for you.

The adverts appear in videos on YouTube channels covering a range of subjects, including pranks, dating, gaming, music and fashion.

They include several by stars such as Adam Saleh whose channel has four million subscribers, and British gamer JMX who has two and a half million subscribers.

Following the BBC's investigation, both have now removed videos with EduBirdie adverts from YouTube.

The BBC also approached the mother of one 12-year-old, who had promoted the company to his 200,000 followers. She also took her advert down.

More time for games

Channels with tens of thousands of subscribers can be offered hundreds of dollars for each advert.

They are not clearly labelled video ads, which are common on YouTube channels.

Instead the YouTuber usually breaks off from what they are doing to personally endorse EduBirdie, promising that the company will deliver an A+ essay for money.

Some YouTubers suggest that using the service will free up time to play video games or take drugs.

So prevalent is the promotion of EduBirdie that very young children are posting videos on YouTube of themselves mimicking the ads.

Sam Gyimah said that EduBirdie's marketing was shocking and pernicious as it presented cheating as "a lifestyle choice".

He said the YouTubers involved should be "called out" for abusing their power as social influencers.

"I think YouTube has a huge responsibility here," he said.

"They do incredibly well from the advertising revenue that they get from the influencers and everyone else. But this is something that is corrosive to education and I think YouTube has got to step up to the plate and exercise some responsibility here."

About 30 of the channels promoting EduBirdie are from Britain and Ireland.

They include a student vlogger at a top UK university.

Another is a popular 15-year-old YouTuber, whose mother was unaware he was promoting the company until she was approached by the BBC.

Shakira Martin, the President of the National Union of Students, said: "I think it's totally disgusting the fact that these type of organisations are exploiting vulnerable young people through getting them to promote something that isn't good, isn't ethical."

She added that students who were working to support themselves while studying might be most tempted to use EduBirdie.

Google's own research found YouTubers were more influential than celebrities when it came to promoting products.

Toni Hopponen, from the tech company Flockler, advises businesses and some universities on how to tap into the power of social influencers.

He said this was creating new challenges as it is outside the regulations that apply to traditional advertising.

"There's always been unethical advertising out there - but now the channels like YouTube provide a way for all of us to be publishers and the scale is huge. "

One British YouTuber, Alpay B urges viewers in one of his videos: "Don't waste your time doing your essays, let these people do it for you."

In a statement, he told the BBC: "Whether a student wants to cheat or not it's totally their choice. You can't really blame EduBirdie or creators who promote them because everyone's got their own hustle."

The BBC ordered two essays through EduBirdie, opting for them to be written from scratch.

One was an English Literature GCSE coursework essay, the other a first-year degree course assignment.

Both were delivered with only the students' names left blank to be filled in.

The GCSE essay was given a C or 5/6 and the university assignment 60% - not quite the guaranteed A+ grade promised by EduBirdie.

Serious consequences

On its website EduBirdie says the essays provided by its writers are "100% plagiarism free".

In practice, this means the essays are written to order, rather than copied and pasted from elsewhere on the internet.

So if a student submits an EduBirdie essay as their own work, it might not be detected by anti-cheating software.

Any university student found to have submitted work done by someone else would face disciplinary action.

"If you've worked hard to get to university, you potentially throw it all away by cheating and getting found out. It is wrong, full stop," Mr Gyimah said.

EduBirdie is run by a company called Boosta, which operates a number of essay-writing websites.

In a statement it said: "We cannot be held responsible for what social influencers say on their channels.

"We give influencers total freedom on how they prefer to present the EduBirdie platform to their audience in a way they feel would be most relevant to their viewers.

"We do admit that many tend to copy and paste each others' shout-outs with a focus on 'get someone to do your homework for you', but this is their creative choice."

It added that there was a disclaimer on the EduBirdie site which suggested that the work it provided should only be used as a sample or a reference.

A YouTube spokesman told the BBC: "YouTube creators may include paid endorsements as part of their content only if the product or service they are endorsing complies with our advertising policies."

They added: "We will be working with creators going forward so they better understand that in video promotions must not promote dishonest activity."

Have you experienced this type of essay buying service? Have you ever paid for an essay? Email haveyoursay@bbc.co.uk.

Please include a contact number if you are willing to speak to a BBC journalist. You can also contact us in the following ways:

• WhatsApp: +44 7555 173285
• Tweet: @BBC_HaveYourSay
• Text an SMS or MMS to 61124 or +44 7624 800 100

The dating feature is likely to be a free service, challenging services such as Match-owned Tinder, which has been rolling out premium, paid features, according to Jefferies analyst Brent Thill.

"It's the first real meaningful competitor," Thill told CNBC's "Power Lunch" following the announcement. "This is a blow to the story [for Match] in the short term."

IAC chief executive Joey Levin said Facebook's product "could be great for US/Russia relationships" but hinted the space had already been cornered.

"Come on in. The water's warm," Levin said in a statement.

Match CEO Mandy Ginsberg echoed the confidence, saying the company was "flattered" Facebook was entering the space.

"We're surprised at the timing given the amount of personal and sensitive data that comes with this territory," Ginsberg said in a statement. "Regardless, we're going to continue to delight our users through product innovation and relentless focus on relationship success. We understand this category better than anyone."

The F8 conference, often an opportunity to announce new developer tools or hardware, comes amid a broader discussion of policy and privacy for the company.

Facebook for months has been dealing with the fallout of the Cambridge Analytica data leak and questions of user privacy — a firestorm set off by reports that an app developer mishandled sensitive user information.

But Zuckerberg said during his address that the company will "keep building, even while we focus on keeping people safe."

The new dating feature links to events and groups on the larger platform, allowing users with dating profiles to connect via shared interests or commonly attended events.

"It mirrors the way people actually date, which is usually at events and institutions that they're connected to," chief product officer Chris Cox said during the F8 conference.

Users can launch text-only private messages, separate from Facebook Messenger or WhatsApp.

The book has much more information than this website, especially political discussion.
The book can be purchased in paperback or digital form at Amazon.com.

Below is the article that has appeared on this website since about 2001:

Cincinnati's Abandoned Subway

Introduction
Abandoned tunnels are often the object of urban legend, but Cincinnati is in fact the site of the country's largest abandoned subway tunnel.  But "abandoned" is not quite the word, as construction slowed to a stop in the late 1920s before even half of the 16 mile line was completed.  Seven miles between Cincinnati's central business district and the industrial suburb of Norwood were tunneled, bridged, or graded, but no track was laid and no subway cars were ordered.  No passengers ever rode between the six stations that were built.

The incomplete Cincinnati line sat fallow through the Great Depression and WWII.  Bridges, stations, and retaining walls along the surface stretches deteriorated to such an extent that a few items actually collapsed.  Nearly everything above ground was bulldozed to make way for portions of I-75 and the Norwood Lateral in the 1950's and 1970's, respectively. The mute two mile tunnel that remains under Central Parkway is unknown to many Cincinnati natives, and what most who do know of it know consists largely of hearsay and speculation.  

This website is the most comprehensive and most accurate source of information regarding the subway either on the web or in print.  It is by far the most popular subject on www.cincinnati-transit.net, and tens of thousands have visited it since its appearance in 1999.
 

Subway F.A.Q.

1. Where is the subway?
The main subway tunnel runs under Central Parkway for two miles, between Walnut St. and an anonymous spot north of the Western Hills Viaduct.  Three underground stations were built and still exist at Race St., Liberty St., and Brighton's Corner.  An extension of this tunnel under Walnut St. south through downtown with a station at Fountain Square was planned but never built.  Additionally, several miles of surface running line were graded and three of roughly a dozen planned above ground stations were built.  Significant portions of today's I-75 and the Norwood Lateral follow the path of the line.  A stretch of I-71 near the Dana Ave. interchange was built where the subway loop's eastern half was planned.

2. When was it constructed?
The main subway tunnel was built in four contracts between 1920 and 1923. A fifth subway contract paid for by the Central Parkway bond issue extended the subway in 1927 north from the Brighton Bridge to the portals still visible from I-75. This means the subway section under Central Parkway forming the north edge of downtown is seven years older than the portals.

3. Can the tunnel still be used?
Yes.  It has been continuously maintained and will likely be usable for the next one hundred years, if not longer.  The 2002 "Metro Moves" sales tax would have funded a rail transit network that planned to use the tunnel, but it was defeated by a 2-1 public vote.

Section 1  Planning and construction
Section 2  Completion attempts
Section 3  The subway today
Section 4  Various proposals
Section 5  What might have been
Section 6  Future use

Construction Photos
Portal Photos
Brighton Station Photos
Linn St. Station Photos
Liberty St. Station Photos
Race St. Station Photos
Hopple St. Tunnel
Norwood Tunnels
1950's Photo Tour
Early Subway Plans and Diagrams
Subway Maps

Back to Cincinnati-Transit.net

webmaster@cincinnati-transit.net
 
 
 
 
 
 
 
 
 
 
 
 

Last week, we received the following email from Amazon:

History

Direct access to Signal has been censored in Egypt, Oman, Qatar, and UAE for the past 1.5 years. These countries attempt to block Signal by blocking connections to Signal servers from all ISPs.

Like most modern services, Signal does not have a single static IP address that ISPs can filter. In cloud environments, IP addresses can change over time as load balancers scale up and down, and the addresses aren’t even always dedicated to a single endpoint. Amazon CloudFront, for example, may terminate requests on the same IP for any number of services that wish to distribute content on their CDN. This can make it more difficult for a censor to identify traffic based on IP address alone.

Unfortunately, a TLS handshake fully exposes the target hostname in plaintext, since the hostname is included in the SNI header in the clear. This remains the case even in TLS 1.3, and it gives a censor all they need.

However, several cloud environments were built with an idiosyncrasy that allowed us to work around this TLS metadata problem. Google and Amazon built their TLS termination layer separately from their request processing layer, such that it was possible to create what looked like a TLS connection for domain A with a request that would actually be received and processed by domain B. This is known as “domain fronting.”

When access to Signal was originally censored in Egypt, Oman, Qatar, and UAE, we responded by deploying domain fronting in those countries through Google App Engine. This meant that to block Signal, those countries would also have to block google.com. That was not a step those countries were willing to take, and as a result Signal has been usable there for the past 1.5 years, even though direct access remains blocked. This required no configuration from users; they could simply install the app and use it as normal.

Iran

Direct access to Signal has also been blocked in Iran for the past 3+ years, but it was not possible to use the same domain fronting technique there. In an apparently unique interpretation of US sanction law, Google does not allow any requests from Iran to be processed by Google App Engine. Requests would get past Iranian censors, but then Google themselves would block them.

In early 2018, a number of policy organizations increased pressure on Google to change their position on how they were interpreting US sanction law so that domain fronting would be possible from Iran. Sadly, these lobbying efforts seem to have had the opposite effect. When Google’s leadership became more aware of domain fronting, it generated internal conversations about whether they wanted to put themselves in the situation of providing cover for sites that entire countries wished to block.

A month later, we received 30-day advance notice from Google that they would be making internal changes to stop domain fronting from working entirely.

AWS

With Google no longer an option, we decided to look for popular domains in censored regions that were on CloudFront instead. Nothing is anywhere near as popular as Google, but there were a few sites that used CloudFront in the Alexa top 50 or 100. We’re an open source project, so the commit switching from GAE to CloudFront was public. Someone saw the commit and submitted it to HN. That post became popular, and apparently people inside Amazon saw it too.

That’s how we got to the above email. Although our interpretation is ultimately not the one that matters, we don’t believe that we are violating the terms they describe:

1. Our CloudFront distribution isn’t using the SSL certificate of any domain but our own.
2. We aren’t falsifying the origin of traffic when our clients connect to CloudFront.

However, in the time-honored tradition of sharing unpopular news late on a Friday afternoon, a few days ago Amazon also announced what they are calling Enhanced Domain Protections for Amazon CloudFront Requests. It is a set of changes designed to prevent domain fronting from working entirely, across all of CloudFront.

Future

With Google Cloud and AWS out of the picture, it seems that domain fronting as a censorship circumvention technique is now largely non-viable in the countries where Signal had enabled this feature. The idea behind domain fronting was that to block a single site, you’d have to block the rest of the internet as well. In the end, the rest of the internet didn’t like that plan.

We are considering ideas for a more robust system, but these ecosystem changes have happened very suddenly. Our team is only a few people, and developing new techniques will take time. Moreover, if recent changes by large cloud providers indicate a commitment to providing network-level visibility into the final destination of encrypted traffic flows, then the range of potential solutions becomes severely limited. If you’d like to help, we’re hiring.

In the meantime, the censors in these countries will have (at least temporarily) achieved their goals. Sadly, they didn’t have to do anything but wait.

Illnesses from mosquito, tick, and flea bites have tripled in the U.S., with more than 640,000 cases reported during the 13 years from 2004 through 2016.  Nine new germs spread by mosquitoes and ticks were discovered or introduced into the United States during this time.

These findings are in the latest Vital Signs report by the Centers for Disease Control and Prevention. This is CDC’s first summary collectively examining data trends for all nationally notifiable diseases caused by the bite of an infected mosquito, tick, or flea. It provides detailed information on the growing burden of mosquito-borne and tickborne illnesses in the U.S.

“Zika, West Nile, Lyme, and chikungunya—a growing list of diseases caused by the bite of an infected mosquito, tick, or flea—have confronted the U.S. in recent years, making a lot of people sick. And we don’t know what will threaten Americans next,” said CDC Director Robert R. Redfield, M.D. “Our Nation’s first lines of defense are state and local health departments and vector control organizations, and we must continue to enhance our investment in their ability to fight against these diseases.”

U.S. not fully prepared

Widespread and difficult to control, diseases from mosquito, tick, and flea bites are major causes of sickness and death worldwide. The growing number and spread of these diseases pose an increasing risk in the U.S. The report found that the nation needs to be better prepared to face this public health threat.

CDC scientists analyzed data reported to the National Notifiable Diseases Surveillance System for 16 notifiable vector-borne diseases from 2004 through 2016 to identify trends. Many infections are not reported or recognized, so it is difficult to truly estimate the overall cost and burden of these diseases. In 2016, the most common tickborne diseases in the U.S. were Lyme disease and ehrlichiosis/anaplasmosis. The most common mosquito-borne viruses were West Nile, dengue, and Zika. Though rare, plague was the most common disease resulting from the bite of an infected flea.

The increase in diseases caused by the bite of an infected mosquito, tick, or flea in the U.S. is likely due to many factors. Mosquitoes and ticks and the germs they spread are increasing in number and moving into new areas. As a result, more people are at risk for infection. Overseas travel and commerce are more common than ever before. A traveler can be infected with a mosquito-borne disease, like Zika, in one country, and then unknowingly transport it home. Finally, new germs spread by mosquito and tick bites have been discovered and the list of nationally notifiable diseases has grown.

Key findings

• A total of 642,602 cases of disease caused by the bite of an infected mosquito, tick, or flea were reported in the U.S. and its territories from 2004 through 2016.
• The number of reported tickborne diseases more than doubled in 13 years and accounted for more than 60 percent of all reported mosquito-borne, tickborne, and fleaborne disease cases. Diseases from ticks vary from region to region across the U.S. and those regions are expanding.
• From 2004 through 2016, seven new germs spread through the bite of an infected tick were discovered or recognized in the U.S. as being able to infect people.
• Reducing the spread of these diseases and responding to outbreaks effectively will require additional capacity at the state and local level for tracking, diagnosing, and reporting cases; controlling mosquitoes and ticks; and preventing new infections; and for the public and private sector to develop new diagnostic and vector control tools.

“The data show that we’re seeing a steady increase and spread of tickborne diseases, and an accelerating trend of mosquito-borne diseases introduced from other parts of the world,” said Lyle Petersen, M.D., M.P.H., director of the Division of Vector-Borne Diseases in the CDC’s National Center for Emerging and Zoonotic Infectious Diseases. “We need to support state and local health agencies responsible for detecting and responding to these diseases and controlling the mosquitoes, ticks, and fleas that spread them.”

How can state and local public health agencies help?

• Build and sustain public health programs that test for and track diseases and the mosquitoes and ticks that spread them.
• Train vector control staff on five core competencies for conducting prevention and control activities. [http://bit.ly/2FG1OMw]
• Educate the public about how to prevent bites and control germs spread by mosquitoes, ticks, and fleas in their communities.

What can everyone do to protect themselves from mosquito, tick, and flea bites?

• Use an Environmental Protection Agency-registered insect repellent. [http://bit.ly/2tIJyLl]
• Wear long-sleeved shirts and long pants.
• Treat items, such as boots, pants, socks, and tents, with permethrin or use permethrin-treated clothing and gear.
• Take steps to control ticks and fleas on pets.
• Find and remove ticks daily from family and pets. [http://bit.ly/2nSlO3S]
• Take steps to control mosquitoes, ticks, and fleas inside and outside your home. [http://bit.ly/2aexzI4 and http://bit.ly/2DbY6E3]

CDC’s efforts to prevent diseases from mosquitoes and ticks

• Funding states and territories to detect and respond to infections from mosquitoes, ticks, and fleas and report cases to CDC. [http://bit.ly/2GvmyDH.]
• Partnering with local and tribal health departments, industry, universities, and international groups to detect and respond to these diseases.
• Supporting five regional centers of excellence to address emerging diseases from mosquitoes and ticks.
• Developing and improving laboratory tests for these diseases.
• Educating the public about protecting themselves from diseases caused by the bite of an infective mosquito, tick, or flea bite.

To read the entire Vital Signs report, visit: www.cdc.gov/vitalsigns/vector-borne/.

For more information about CDC’s work on vector-borne diseases, please visit: www.cdc.gov/vector

About Vital Signs

Vital Signs is a report that appears as part of the CDC’s Morbidity and Mortality Weekly Report. Vital Signs provides the latest data and information on key health threats. Some topics include cancer, HIV/AIDS, prescription drug overdoses, antibiotic resistance, suicides, asthma, and global health

###
U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES

What's in those mysterious cabinets?

Last Monday some folks were working on this thing on Walnut Street. I didn't remember having seen the inside of one before, so I took some pictures of it to look at later.

Thanks to the Wonders of the Internet, it didn't take long to figure out what it is for. It is a controller for the traffic lights at the intersection.

In particular, the top module in the right-hand picture is a Model 170 ATC HC11 Controller manufactured by McCain Inc, a thirty-year old manufacturer of traffic control devices. The controller runs software developed and supported by McCain, and the cabinet is also made by McCain.

The descriptions of the controllers are written in a dense traffic control jargon that I find fascinating but opaque. For example, the 170 controller's product description reads:

The McCain 170E, 170E HC11, and 170 ATC HC11 controllers’ primary design function is to operate eight-phase dual ring intersections. Based on the software control package utilized, the 170’s control applications can expand to include: ramp metering, variable message signs, sprinklers, pumps, and changeable lane control.

I think I understand what variable message signs are, and I can guess at changeable lane control, but what are the sprinklers and pumps for? What is ramp metering?

The eight-phase dual ring intersection, which I had never heard of before, is an important topic in the traffic control world. I gather that it is a four-way intersection with a four-way traffic light that also has a left-turn-only green arrow portion. The eight “phases” refer to different traffic paths through the intersections that must be separately controlled: even numbers for the four paths through the intersection, and odd numbers 1,3,5,7 for the left-turn-only paths that do not pass through. Some phases conflict; for example phase 5 (left-turning in some direction, say from south to east), conflicts with phase 6 (through-passing heading in the opposite direction) but not with phase 1 (left-turning from north to west).

There's plenty of detailed information about this available. For example, the U.S. Federal Highway Administration publishes theirTraffic Signal Timing Manual. (Published in 2008, it has since been superseded.) Unfortunately, this seems to be too advanced for me! Section 4.2.1 (“Definitions and Terminology”) is the first place in the document that mentions the dual-ring layout, and it does so without explanation — apparently this is so elementary that anyone reading the Traffic Signal Timing Manual will already be familiar with it:

Over the years, the description of the “individual movements” of the dual-ring 8-movement controller as “phases” has blurred into common communicated terminology of “movement number” being synonymous as “phase number”.

But these helpful notes explain in more detail: a “ring” is “a sequence of phases that are not compatible and that must time sequentially”.

Then we measure the demand for each phase, and there is an interesting and complex design problem: how long should each phase last to optimize traffic flow through the intersection for safety and efficiency? See chapter 3a for more details of how this is done.

I love when I discover there is an entire technical domain that I never even suspected existed. If you like this kind of thing, you may enjoy geeking out over the Manual of Uniform Traffic Control Devices, which explains what traffic signs should look like and what each one means. Have you ever noticed that the green guide signs on the highway have up-pointing and down-pointing arrows that are totally different shapes?

That's because they have different meanings: the up-pointing arrows mean “go this way” and the down-pointing arrows mean “use this lane”. The MUTCD says what the arrows should look like, how big they should be, and when each one should be used.

The MUTCD is the source of one of my favorite quotations:

Regulatory and warning signs should be used conservatively because these signs, if used to excess, tend to lose their effectiveness.

Words to live by! Programmers in particular should keep this in mind when designing error messages. You could spend your life studying this 864-page manual, and I think some people do.

Related geekery: Geometric highway design: how sharply can the Interstate curve and still be safe, and how much do the curves need to be banked? How do you design an interchange between two major highways? How about a highway exit?

Here's a highway off-ramp, exit 346A on Pennsylvania I76 West:

Did you know that the long pointy triangle thing is called a “gore”?

What happens if you can't make up your mind whether to stay on the highway or take the exit, you drive over the gore, and then smack into the thing beyond it where the roads divides? Well, you might survive, because there is a thing there that is designed to crush when you hit it. It might be a QuadGuard Elite Crash Cushion System, manufactured by Energy Absorption Systems, Inc..

It's such a big world out there, so much to know.

[Other articles in category /tech] permanent link

Two and a half years ago, a team of researchers led by groups at MIT, the University of California at Berkeley, and Boston University announced a milestone: the fabrication of a working microprocessor, built using only existing manufacturing processes, that integrated electronic and optical components on the same chip.

The researchers’ approach, however, required that the chip’s electrical components be built from the same layer of silicon as its optical components. That meant relying on an older chip technology in which the silicon layers for the electronics were thick enough for optics.

In the latest issue of Nature, a team of 18 researchers, led by the same MIT, Berkeley, and BU groups, reports another breakthrough: a technique for assembling on-chip optics and electronic separately, which enables the use of more modern transistor technologies. Again, the technique requires only existing manufacturing processes.

“The most promising thing about this work is that you can optimize your photonics independently from your electronics,” says Amir Atabaki, a research scientist at MIT’s Research Laboratory of Electronics and one of three first authors on the new paper. “We have different silicon electronic technologies, and if we can just add photonics to them, it’d be a great capability for future communications and computing chips. For example, now we could imagine a microprocessor manufacturer or a GPU manufacturer like Intel or Nvidia saying, ‘This is very nice. We can now have photonic input and output for our microprocessor or GPU.’ And they don’t have to change much in their process to get the performance boost of on-chip optics.”

Light appeal

Moving from electrical communication to optical communication is attractive to chip manufacturers because it could significantly increase chips’ speed and reduce power consumption, an advantage that will grow in importance as chips’ transistor count continues to rise: The Semiconductor Industry Association has estimated that at current rates of increase, computers’ energy requirements will exceed the world’s total power output by 2040.

The integration of optical — or “photonic” — and electronic components on the same chip reduces power consumption still further. Optical communications devices are on the market today, but they consume too much power and generate too much heat to be integrated into an electronic chip such as a microprocessor. A commercial modulator — the device that encodes digital information onto a light signal — consumes between 10 and 100 times as much power as the modulators built into the researchers’ new chip.

It also takes up 10 to 20 times as much chip space. That’s because the integration of electronics and photonics on the same chip enables Atabaki and his colleagues to use a more space-efficient modulator design, based on a photonic device called a ring resonator.

“We have access to photonic architectures that you can’t normally use without integrated electronics,” Atabaki explains. “For example, today there is no commercial optical transceiver that uses optical resonators, because you need considerable electronics capability to control and stabilize that resonator.”

Atabaki’s co-first-authors on the Nature paper are Sajjad Moazeni, a PhD student at Berkeley, and Fabio Pavanello, who was a postdoc at the University of Colorado at Boulder, when the work was done. The senior authors are Rajeev Ram, a professor of electrical engineering and computer science at MIT; Vladimir Stojanovic, an associate professor of electrical engineering and computer sciences at Berkeley; and Milos Popovic, an assistant professor of electrical and computer engineering at Boston University. They’re joined by 12 other researchers at MIT, Berkeley, Boston University, the University of Colorado, the State University of New York (SUNY) Polytechnic Institute, and Ayar Labs, an integrated-photonics startup that Ram, Stojanovic, and Popovic helped found.

Sizing crystals

In addition to millions of transistors for executing computations, the researchers’ new chip includes all the components necessary for optical communication: modulators; waveguides, which steer light across the chip; resonators, which separate out different wavelengths of light, each of which can carry different data; and photodetectors, which translate incoming light signals back into electrical signals.

Silicon — which is the basis of most modern computer chips — must be fabricated on top of a layer of glass to yield useful optical components. The difference between the refractive indices of the silicon and the glass — the degrees to which the materials bend light — is what confines light to the silicon optical components.

The earlier work on integrated photonics, which was also led by Ram, Stojanovic, and Popovic, involved a process called wafer bonding, in which a single, large crystal of silicon is fused to a layer of glass deposited atop a separate chip. The new work, in enabling the direct deposition of silicon — with varying thickness — on top of glass, must make do with so-called polysilicon, which consists of many small crystals of silicon.

Single-crystal silicon is useful for both optics and electronics, but in polysilicon, there’s a tradeoff between optical and electrical efficiency. Large-crystal polysilicon is efficient at conducting electricity, but the large crystals tend to scatter light, lowering the optical efficiency. Small-crystal polysilicon scatters light less, but it’s not as good a conductor.

Using the manufacturing facilities at SUNY Polytechnic Institute's Colleges of Nanoscale Sciences and Engineering, the researchers tried out a series of recipes for polysilicon deposition, varying the type of raw silicon used, processing temperatures and times, until they found one that offered a good tradeoff between electronic and optical properties.

“I think we must have gone through more than 50 silicon wafers before finding a material that was just right,” Atabaki says.

Ingrid Newkirk, co-founder and president of People for the Ethical Treatment of Animals, flew to Silicon Valley earlier this year for a long-planned speech to Google employees. It wasn’t until she sat waiting in a parking lot that a call came through notifying her the event was canceled.

Ms. Newkirk had been invited by some employees to discuss her view that animals can be subject to prejudice just as people can, as part of the company’s “Talks at Google” series. Another group of employees said the topic was offensive to humans who face racism, and they protested.

There are several issues in drivers/char/random.c, in particular related to the
behavior of the /dev/urandom RNG during and shortly after boot.

I'm sending this to security@kernel.org and Theodore Ts'o for now; it might make
sense to also add Jason Donenfeld, since he's done some work around boot
randomness?

== Discarded early randomness, including device randomness ==
A comment above rand_initialize() explains:

/*
 * Note that setup_arch() may call add_device_randomness()
 * long before we get here. This allows seeding of the pools
 * with some platform dependent data very early in the boot
 * process. But it limits our options here. We must use
 * statically allocated structures that already have all
 * initializations complete at compile time. We should also
 * take care not to overwrite the precious per platform data
 * we were given.
 */

In other words, the intent is that none of the early randomness, in particular
device randomness, should be discarded.

rand_initialize() starts by "initializing" the input_pool and the blocking_pool
by mixing some extra entropy into them (real time, multiple time stamp counters
and the utsname); it doesn't clear the pools to avoid clobbering existing
entropy.
The primary_crng, however, is fully reinitialized, discarding its existing
state.

In the crng_init==0 stage, entropy from various in-kernel sources, including
device randomness and interrupt randomness, is fed into the primary_crng
directly, but not into the input_pool.

Therefore, the entropy that was collected in the crng_init==0 stage will
disappear during rand_initialize().

AFAICS device randomness is discarded since
commit ee7998c50c26 ("random: do not ignore early device randomness"); before
that, only interrupt randomness and hardware generator randomness were discarded
this way.

== RNG is treated as cryptographically safe too early ==
Multiple callers, including sys_getrandom(..., flags=0), attempt to wait for the
RNG to become cryptographically safe before reading from it by checking for
crng_ready() and waiting if necessary. However, crng_ready() only checks for
`crng_init > 0`, and `crng_init==1` does not imply that the RNG is
cryptographically safe.

Interrupt randomness is mixed in a fast pool of size 16 bytes, and every 64
interrupts, the fast pool is flushed into the primary_crng. That's 1/4 byte per
interrupt in the fast load accounting.
OTOH, device randomness is piped straight into the primary_crng and accounted
with one byte per written byte.
As soon as 64 bytes have been written into the primary_crng, the RNG moves to
crng_init==1.
This accounting is very unbalanced.

The device entropy fed into the kernel in this way includes:

 - DMI table
 - kernel command line string
 - MAC addresses of network devices
 - USB device serial, product, and manufacturers (all as strings)

On a system I'm testing on, in practice, the RNG just reads the DMI table and
then, since the DMI table is way bigger than 64 bytes, immediately moves to
crng_init==1 without using even a single sample of interrupt randomness.

The worst part of this (one device entropy sample being enough to move to
crng_init==1) was AFAICS introduced in
commit ee7998c50c26 ("random: do not ignore early device randomness"), first in
v4.14.

== Interaction between kernel and entropy-persisting userspace is broken ==
A comment above the kernel code suggests:

 * Ensuring unpredictability at system startup
 * ============================================
 *
 * When any operating system starts up, it will go through a sequence
 * of actions that are fairly predictable by an adversary, especially
 * if the start-up does not involve interaction with a human operator.
 * This reduces the actual number of bits of unpredictability in the
 * entropy pool below the value in entropy_count.  In order to
 * counteract this effect, it helps to carry information in the
 * entropy pool across shut-downs and start-ups.  To do this, put the
 * following lines an appropriate script which is run during the boot
 * sequence:
 *
 * echo "Initializing random number generator..."
 * random_seed=/var/run/random-seed
 * # Carry a random seed from start-up to start-up
 * # Load and then save the whole entropy pool
 * if [ -f $random_seed ]; then
 * cat $random_seed >/dev/urandom
 * else
 * touch $random_seed
 * fi
 * chmod 600 $random_seed
 * dd if=/dev/urandom of=$random_seed count=1 bs=512
 *
 * and the following lines in an appropriate script which is run as
 * the system is shutdown:
[...]
 * Effectively, these commands cause the contents of the entropy pool
 * to be saved at shut-down time and reloaded into the entropy pool at
 * start-up.  (The 'dd' in the addition to the bootup script is to
 * make sure that /etc/random-seed is different for every start-up,
 * even if the system crashes without executing rc.0.)  Even with
 * complete knowledge of the start-up activities, predicting the state
 * of the entropy pool requires knowledge of the previous history of
 * the system.

Counterintuitively, after such a startup script has executed, the seed data
reloaded by the script probably won't actually influence data that is read from
/dev/urandom directly afterwards:

 - If the seed data is loaded with crng_init < 2, the seed data written into the
   input_pool will not flow into the primary_crng or into the NUMA CRNGs until
   `crng_init == 2`.
 - If the seed data is loaded with `crng_init == 2`, the seed data written into
   the input_pool will only propagate into the primary_crng, and from there into
   the NUMA CRNGs, with delays of 5 minutes (!) each (CRNG_RESEED_INTERVAL).

This has two consequences:

 - Services that seed their own RNG from /dev/urandom shortly after the seed
   data has been loaded into the kernel RNG will probably only use boot entropy;
   the RNG seeds used by such services will be independent from the persistent
   seed.
 - The data written back to the seed file by the boot script will be independent
   from the previous persistent seed; if the system is shut down uncleanly
   (without running the shutdown script) and then powered up again, the
   persistent seed file will only contain entropy collected during the previous
   boot.


== No entropy is fed into NUMA CRNGs between rand_initialize() initcall and crng_init==2 ==
When the RNG subsystem is initialized using the early_initcall hook
rand_initialize, the NUMA CRNGs (introduced in
commit 1e7f583af67b ("random: make /dev/urandom scalable for silly userspace programs"),
first in v4.8) are initialized using entropy from the primary_crng after it has
been reinitialized from the input_pool. This entropy is:

 - If crng_init==0: Real time, some cycle counters, utsname (all from
   init_std_data() and crng_initialize()), and potentially events from
   add_timer_randomness() if any have happened at that point.
 - If crng_init==1: Real time, some cycle counters, utsname, all timer
   randomness that has happened up to the rand_initialize() call, and any
   device/timer/hardware-rng/interrupt randomness that may have come in between
   the time crng_init became 1 and the rand_initialize() call, and are not still
   batched.

In the crng_init==0 case, the primary_crng will be fed with entropy until
crng_init==1; but in either case, no more entropy can reach the NUMA CRNGs until
crng_init==2, even though the kernel will assume that the NUMA CRNGs are
cryptographically safe once crng_init==1.

In other words, /dev/urandom reads will return data whose entropy only comes
from timing samples in the first few dozen milliseconds of system boot for
(depending on the system) minutes after the system has booted.


== initcall can propagate entropy into primary and NUMA CRNGs while crng_init==1 ==
My understanding of the intent behind the crng_init states is as follows:

 - state 0: early startup; want to get entropy into the RNG quickly
 - state 1: buffer up 128 bits of entropy to prevent an attacker with access
   to multiple RNG samples across system boot from continuously brute-forcing
   the RNG input in small chunks
 - state 2: feed all the buffered entropy into the RNG at once, then continue
   feeding entropy into the RNG every 5 minutes

If this interpretation is correct, it is problematic that, if the
rand_initialize() initcall happens while crng_init==1, entropy from the input
pool is propagated into the primary RNG and the NUMA CRNGs: If this happens, the
amount of entropy that is fed into the user-accessible RNGs at once is, in the
theoretical worst case, halved.


== Impact ==
I have spent a few days attempting to figure out how bad these issues are.
I believe that on an Intel Grass Canyon system, with RDRAND disabled,
ASLR disabled, fast boot enabled, no connected devices, with boot on power,
some frequency scaling options disabled, and the fan set to maximum,
it should be possible to express the entropy in the used RDTSC samples in around
105 bits or less. (I'm not sure which parts of this configuration actually
influence the amount of entropy; but ASLR certainly does influence it, since the
one interrupt sample that is fed into the RNG before the RNG initialization
contains an instruction pointer.)

From eight boots, the initial TSC samples (in hex):
11ea2f6f6,11ea54523,11e6337b9,11ea1100c,11e9e66d6,11e9d5165,11e7d1742,11e9e4a9d

The deltas between following TSC samples (in hex; each block of numbers
corresponds to one boot):

479a b214a34 3021c16 9fccbb d7 7d 6e 69 73 69 69 69 69 69 69 69 73 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 6e 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 51c7 a a a a a a 5 a a a 5

47b8 b205fb6 3025a4b 9fd990 dc 7d 69 69 73 69 69 69 69 69 69 69 73 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 6e 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 519a f a a a a a 5 a a a 5

479a b23b02b 3023930 9f89f9 d7 7d 6e 69 73 69 69 69 69 69 69 69 73 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 6e 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 523a f a a a a a 5 a a a 5

47b3 b2053b8 30223be 9fc76b dc 7d 69 69 73 69 69 69 69 69 69 69 73 69 69 69 69 69 69 69 69 69 69 69 69 69 6e 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 51e0 a a a a a a 5 a a a 5

4565 b2096ac 3021b30 9fa22c d2 7d 6e 69 73 69 69 69 69 69 69 69 73 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 6e 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 5208 f a 5 a a a a a 5 a a

47ae b20cab4 301e7d2 9fb82a d2 7d 6e 69 6e 69 69 69 69 69 6e 69 73 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 6e 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 51ea a a a a a a a 5 a a a

4795 b21227f 30218e2 9ffe66 d2 7d 6e 69 6e 69 69 69 69 69 6e 69 73 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 6e 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 551e f 5 a a a a a 5 a a a

4795 b2242bd 30230fc 9fb6ae d7 7d 69 69 73 69 69 69 69 69 69 69 73 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 6e 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 69 5140 f a a a 5 a a a a 5 a

On top of that, there is entropy from the ktime_get_real() call in
init_std_data(); the amount of entropy from that depends on how precisely an
attacker knows the system boot time.


This bug is subject to a 90 day disclosure deadline. After 90 days elapse
or a patch has been made broadly available, the bug report will become
visible to the public.

Individuals seeking employment at Nurx are considered without regards to race, color, religion, national origin, age, sex, marital status, ancestry, physical or mental disability, veteran status, gender identity, or sexual orientation. You are being given the opportunity to provide the following information in order to help us comply with federal and state Equal Employment Opportunity/Affirmative Action record keeping, reporting, and other legal requirements.

Completion of the form is entirely voluntary. Whatever your decision, it will not be considered in the hiring process or thereafter. Any information that you do provide will be recorded and maintained in a confidential file.

Gender

Are you Hispanic/Latino?

Please identify your race

If you believe you belong to any of the categories of protected veterans listed below, please indicate by making the appropriate selection. As a government contractor subject to Vietnam Era Veterans Readjustment Assistance Act (VEVRAA), we request this information in order to measure the effectiveness of the outreach and positive recruitment efforts we undertake pursuant to VEVRAA. Classification of protected categories is as follows:

A "disabled veteran" is one of the following: a veteran of the U.S. military, ground, naval or air service who is entitled to compensation (or who but for the receipt of military retired pay would be entitled to compensation) under laws administered by the Secretary of Veterans Affairs; or a person who was discharged or released from active duty because of a service-connected disability.

A "recently separated veteran" means any veteran during the three-year period beginning on the date of such veteran's discharge or release from active duty in the U.S. military, ground, naval, or air service.

An "active duty wartime or campaign badge veteran" means a veteran who served on active duty in the U.S. military, ground, naval or air service during a war, or in a campaign or expedition for which a campaign badge has been authorized under the laws administered by the Department of Defense.

An "Armed forces service medal veteran" means a veteran who, while serving on active duty in the U.S. military, ground, naval or air service, participated in a United States military operation for which an Armed Forces service medal was awarded pursuant to Executive Order 12985.

Veteran Status

Because we do business with the government, we must reach out to, hire, and provide equal opportunity to qualified people with disabilities¹. To help us measure how well we are doing, we are asking you to tell us if you have a disability or if you ever had a disability. Completing this form is voluntary, but we hope that you will choose to fill it out. If you are applying for a job, any answer you give will be kept private and will not be used against you in any way.

If you already work for us, your answer will not be used against you in any way. Because a person may become disabled at any time, we are required to ask all of our employees to update their information every five years. You may voluntarily self-identify as having a disability on this form without fear of any punishment because you did not identify as having a disability earlier.

You are considered to have a disability if you have a physical or mental impairment or medical condition that substantially limits a major life activity, or if you have a history or record of such an impairment or medical condition.

Disabilities include, but are not limited to:

• Blindness
• Deafness
• Cancer
• Diabetes
• Epilepsy
• Autism
• Cerebral palsy
• HIV/AIDS
• Schizophrenia
• Muscular dystrophy
• Bipolar disorder
• Major depression
• Multiple sclerosis (MS)
• Missing limbs or partially missing limbs
• Post-traumatic stress disorder (PTSD)
• Obsessive compulsive disorder
• Impairments requiring the use of a wheelchair
• Intellectual disability (previously called mental retardation)

Disability Status

Federal law requires employers to provide reasonable accommodation to qualified individuals with disabilities. Please tell us if you require a reasonable accommodation to apply for a job or to perform your job. Examples of reasonable accommodation include making a change to the application process or work procedures, providing documents in an alternate format, using a sign language interpreter, or using specialized equipment.

¹Section 503 of the Rehabilitation Act of 1973, as amended. For more information about this form or the equal employment obligations of Federal contractors, visit the U.S. Department of Labor's Office of Federal Contract Compliance Programs (OFCCP) website at www.dol.gov/ofccp.

PUBLIC BURDEN STATEMENT: According to the Paperwork Reduction Act of 1995 no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. This survey should take about 5 minutes to complete.

A dozen Indian workers at an East Bay technology firm were promised salaries of up to $8,300 a  month, but after the company brought them in under the controversial H-1B visa, they found themselves netting as little as $800 per month, the federal government alleged Tuesday.

Cloudwick Technologies of Newark has been ordered to pay about $175,000 to 12 employees for back wages after violating H-1B rules, the U.S. Department of Labor said. The company disputed some of the government’s claims.

“Investigators found that the company paid impacted employees well below the wage levels required under the H-1B program based on job skill level, and also made illegal deductions from workers’ salaries,” the department said in a news release. “As a result, some of the H-1B employees that Cloudwick brought from India with promised salaries of up to $8,300 per month instead received as little as $800 net per month.”

Cloudwick founder and CEO Mani Chhabra said Tuesday that the labor department “misrepresented” some facts about the company’s use of H-1B workers.

“Cloudwick has never brought resources from India,” Chhabra said. “All the resources are Master’s students that have educated in U.S. and then we hired and trained them.”

The back-wages issue arose because of changes in visa categories imposed by the labor department, he said. Workers were paid $800 a month during training, he said. He acknowledged that the company improperly deducted money for four workers for further training.

“This is not allowed for H-1B and we paid them back wages,” Chhabra said.

A labor department spokesman declined to specify how the alleged visa abuse came to light. Many investigations start with complaints, which are confidential, the spokesman said.

The department also did not say how long the workers had been employed at Cloudwick, but the federal probe ran from the start of 2015 to the end of last year.

The H-1B visa, intended for jobs requiring specialized skills and a bachelor’s degree or higher, has become a flashpoint in the immigration debate, with critics pointing to alleged abuses in which American workers at UC San Francisco and Disney were reportedly forced to train Indian replacement workers. Major technology firms have lobbied aggressively for expansion of the program, arguing that they need access to the world’s top talent to fill highly-technical jobs.

News of the crackdown on Cloudwick will likely fuel arguments that the H-1B program is being abused by companies to get cheap foreign labor at the expense of jobs for Americans.

Cloudwick — a data-analytics company whose clients include Apple, Cisco, Comcast, American Express, Bank of America, Safeway, Verizon and Visa — agreed in writing to hire an independent monitor to ensure future compliance with the H-1B rules, the labor department said.

U.S. government data shows Cloudwick received 27 approvals for H-1B applications last year, and 55 in 2016.

The labor department declined to say whether it believed other H-1B workers at Cloudwick were being paid what they were promised, and in keeping with visa rules. Cloudwick did not respond to a question about salaries for its other H-1B workers.

There are three broad concurrency control techniques, i.e.Multi-version Concurrency Control (MVCC),Strict Two-Phase Locking (S2PL), and Optimistic Concurrency Control (OCC), and each technique has many variations. In MVCC, each write operation creates a new version of a data item while retaining the old version. When a transaction reads a data item, the system selects one of the versions to ensure isolation of the individual transaction. The main advantage of MVCC is that 'readers don’t block writers, and writers don’t block readers', in contrast, for example, an S2PL-based system must block readers when a writer writes an item because the writer acquires an exclusive lock for the item. PostgreSQL and some RDBMSs use a variation of MVCC called Snapshot Isolation (SI).

To implement SI, some RDBMSs, e.g., Oracle, use rollback segments. When writing a new data item, the old version of the item is written to the rollback segment, and subsequently the new item is overwritten to the data area. PostgreSQL uses a simpler method. A new data item is inserted directly into the relevant table page. When reading items, PostgreSQL selects the appropriate version of an item in response to an individual transaction by applying visibility check rules.

SI does not allow the three anomalies defined in the ANSI SQL-92 standard, i.e. Dirty Reads, Non-Repeatable Reads, and Phantom Reads. However, SI cannot achieve true serializability because it allows serialization anomalies, such as Write Skew and Read-only Transaction Skew. Note that the ANSI SQL-92 standard based on the classical serializability definition is not equivalent to the definition in modern theory. To deal with this issue, Serializable Snapshot Isolation (SSI) has been added as of version 9.1. SSI can detect the serialization anomalies and can resolve the conflicts caused by such anomalies. Thus, PostgreSQL version 9.1 and later provides a true SERIALIZABLE isolation level. (In addition, SQL Server also uses SSI, Oracle still uses only SI.)

This chapter comprises the following four parts:

• Part 1: Sections 5.1. — 5.3.

This part provides basic information required for understanding the subsequent parts.

Sections 5.1 and 5.2 describe transaction ids and tuple structure, respectively. Section 5.3 exhibits how tuples are inserted, deleted, and updated.

• Part 2: Sections 5.4. — 5.6.

This part illustrates the key features required for implementing the concurrency control mechanism.

Sections 5.4, 5.5, and 5.6 describe the commit log (clog), which holds all transaction states, transaction snapshots, and the visibility check rules, respectively.

• Part 3: Sections 5.7. — 5.9.

This part describes the concurrency control in PostgreSQL using specific examples.

Section 5.7 describes the visibility check. This section also shows how the three anomalies defined in the ANSI SQL standard are prevented. Section 5.8 describes preventing Lost Updates, and Section 5.9 briefly describes SSI.

• Part 4: Section 5.10.

This part describes several maintenance process required to permanently running the concurrency control mechanism. The maintenance processes are performed by vacuum processing, which is described in Chapter 6.

This chapter focuses on the topics that are unique to PostgreSQL, although there are many concurrency control-related topics. Note that descriptions of deadlock prevention and lock modes are omitted (refer to the official documentation for more information).

Over the past year, Kubernetes––also known as K8s––has become a dominant topic of conversation in the infrastructure world. Given its pedigree of literally working at Google-scale, it makes sense that people want to bring that kind of power to their DevOps stories; container orchestration turns many tedious and complex tasks into something as simple as a declarative config file.

The rise of orchestration is predicated on a few things, though. First, organizations have moved toward breaking up monolithic applications into microservices. However, the resulting environments have hundreds (or thousands) of these services that need to be managed. Second, infrastructure has become cheap and disposable––if a machine fails, it’s dramatically cheaper to replace it than triage the problems.

So, to solve the first issue, orchestration relies on the boon of the second; it manages services by simply letting new machines, running the exact same containers, take the place of failed ones, which keeps a service running without any manual interference.

However, the software most amenable to being orchestrated are ones that can easily spin up new interchangeable instances without requiring coordination across zones.

Why Orchestrating Databases is Difficult

The above description of an orchestration-native service should sound like the opposite of a database, though.

• Database replicas are not interchangeable; they each have a unique state. This means you cannot trivially bring them up and down at a moment’s notice.
• Deploying a database replica requires coordination with other nodes running the same application to ensure things like schema changes and version upgrades are visible everywhere.

In short: managing state in Kubernetes is difficult because the system’s dynamism is too chaotic for most databases to handle––especially SQL databases that offer strong consistency.

Running a Database with a Kubernetes App

So, what’s a team to do? Well, you have a lot of options.

Run Your Database Outside Kubernetes

Instead of running your entire stack inside K8s, one approach is to continue to run the database outside Kubernetes. The main challenge with this, though, is that you must continue running an entire stack of infrastructure management tools for a single service. This means that even though Kubernetes has a high-quality, automated version of each of the following, you’ll wind up duplicating effort:

• Process monitoring (monit, etc.)
• Configuration management (Chef, Puppet, Ansible, etc.)
• In-datacenter load balancing (HAProxy)
• Service discovery (Consul, Zookeeper, etc.)
• Monitoring and logging

That’s 5 technologies you’re on the hook for maintaining, each of which is duplicative of a service already integrated into Kubernetes.

Cloud Services

Rather than deal with the database at all, you can farm out the work to a database-as-a-service (DBaaS) provider. However, this still means that you’re running a single service outside of Kubernetes. While this is less of a burden, it is still an additional layer of complexity that could be instead rolled into your teams’ existing infrastructure.

For teams that are hosting Kubernetes themselves, it’s also strange to choose a DBaaS provider. These teams have put themselves in a situation where they could easily avoid vendor lock-in and maintain complete control of their stack.

DBaaS offerings also have their own shortcomings, though. The databases that underpin them are either built on dated technology that doesn’t scale horizontally, or require forgoing consistency entirely by relying on a NoSQL database.

Run Your Database in K8s––StatefulSets & DaemonSets

Kubernetes does have two integrated solutions that make it possible to run your database in Kubernetes:

StatefulSets

By far the most common way to run a database, StatefulSets is a feature fully supported as of the Kubernetes 1.9 release. Using it, each of your pods is guaranteed the same network identity and disk across restarts, even if it’s rescheduled to a different physical machine.

DaemonSets

DaemonSets let you specify that a group of nodes should always run a specific pod. In this way, you can set aside a set of machines and then run your database on them––and only your database, if you choose. This still leverages many of Kubernetes’ benefits like declarative infrastructure, but it forgoes the flexibility of a feature like StatefulSets that can dynamically schedule pods.

StatefulSets: In-Depth

StatefulSets were designed specifically to solve the problem of running stateful, replicated services inside Kubernetes. As we discussed at the beginning of this post, databases have more requirements than stateless services, and StatefulSets go a long way to providing that.

The primary feature that enables StatefulSets to run a replicated database within Kubernetes is providing each pod a unique ID that persists, even as the pod is rescheduled to other machines. The persistence of this ID then lets you attach a particular volume to the pod, retaining its state even as Kubernetes shifts it around your datacenter.

However, because you’ll be detaching and attaching the same disk to multiple machines, you need to use a remote persistent disk, something like EBS in AWS parlance. These disks are located––as you might guess––remotely from any of the machines and are typically large block devices used for persistent storage. One of the benefits of using these disks is that the provider handles some degree of replication for you, making them more immune to typical disk failures, though this benefits databases without built-in replication.

Performance Implications

Because Kubernetes itself runs on the machines that are running your databases, it will consume some resources and will slightly impact performance. In our testing, we found an approximately 5% dip in throughput on a simple key-value workload.

Because StatefulSets still let your database pods to be rescheduled onto other nodes, it’s possible that the stateful service will still have to contend with others for the machine’s physical resources. However, you can take steps to alleviate this issue by managing the resources that the database container requests.

DaemonSets: In-Depth

DaemonSets let you specify that all nodes that match a specific criteria run a particular pod. This means you can designate a specific set of nodes to run your database, and Kubernetes ensures that the service stays available on these nodes without being subject to rescheduling––and optionally without running anything else on those nodes, which is perfect for stateful services.

DaemonSets can also use a machine’s local disk more reliably because you don’t have to be concerned with your database pods getting rescheduled and losing their disks. However, local disks are unlikely to have any kind of replication or redundancy and are therefore more susceptible to failure, although this is less of a concern for services like CockroachDB which already replicate data across machines.

Performance Implications

While some K8s processes still run on these machines, DaemonSets can limit the amount of contention between your database and other applications by simply cordoning off entire Kubernetes nodes.

StatefulSets vs. DaemonSets

Kubernetes StatefulSets behave like all other Kubernetes pods, which means they can be rescheduled as needed. Because other types of pods can also be rescheduled onto the same machines, you’ll also need to set appropriate limits to ensure your database pods always have adequate resources allocated to them.

StatefulSets’ reliance on remote network devices also means there is a potential performance implication, though in our testing, this hasn’t been the case.

DaemonSets on the other hand, are dramatically different. They represent a more natural abstraction for cordoning your database off onto dedicated nodes and let you easily use local disks––for StatefulSets, local disk support is still in beta.

The biggest tradeoff for DaemonSets is that you’re limiting Kubernetes’ ability to help your cluster recover from failures. For example, if you were running CockroachDB and a node were to fail, it can’t create new pods to replace pods on nodes that fail because it’s already running a CockroachDB pod on all the matching nodes. This matches the behavior of running CockroachDB directly on a set of physical machines that are only manually replaced by human operators.

Up Next

In our next blog post, we’ll continue talking about stateful applications on Kubernetes, with details about how you can can (and should) orchestrate CockroachDB in Kubernetes leveraging StatefulSets. If you want to be notified when it’s released, subscribe to our blog using the box on the left.

In the meantime, you should check out our Kubernetes tutorial.

Illustration by Zoë van Dijk

I actually started this RPi conversion first, but I got sidetracked and ended up finishing the Amiga 500 one first. More on that here.

So the objective here was to take a C64 breadbin case and keyboard and put a Raspberry Pi 3 into it; keeping the keyboard and joystick ports working, but also giving me HDMI, USB controller support, and modem emulation. While I still have 2 real Commodore 64s (and an Ultimate64 on the way!), I like using the RPi and Vice to play 64 games.

These mounts do not require you to drill or cut your C64 case! The 3D files are provisioned under the creative commons license so they are FREE to use, distribute, modify, or even sell.

Pictures

Everything in its place.

Beautiful backside -- from right to left -- micro USB power, power switch, HDMI.

Outside shot of the Keyrah.

Instructions

1. Print out the 3D parts. You will have left, middle, right, and 3 washer pieces.
2. Put the 4 heat sets into the mounting holes of the left piece. Use a soldering iron to heat them into place.
3. Screw in the micro USB and HDMI panel mount cables into the heat sets.
4. Snap the left piece into the back. Screw the 2 washers into place to hold that piece down.
5. Put the 4 heat sets into the RPi mounting holes of the middle piece. Place it into the case and use the 3rd washer to screw it to the case.
6. Mount the RPi onto the middle piece as shown in the case.
7. Place the right piece onto the Keyrah -- note how the piece "wraps" the external USB port of the Keyrah. Place into the case and screw down. You do NOT need to screw the bracket into case, but can if you wish.
8. Place the power switch in the left piece and connect to the PowerBlock. Connect the keyboard to the Keyrah, and the LED to the PowerBlock. Plug the PowerBlock into the RPi.
9. Turn on!

Parts List

(1) Raspberry Pi 3B
(1) Micro SD card
(1) Keyrah v2
(1) PowerBlock
(1) Micro USB male to female panel mount cable
(1) HDMI panel mount cable
(1) 14x9mm power switch
(4) M2 x 5mm screws
(8) M2 x 3mm brass heat sets
(8) M2 5mm x 1mm flat insulating washers
(1) Header pins for the internal Keyrah USB -- you will need to solder these to the Keyrah
(1) USB-A to header cable for connecting the Keyrah to the RPi
(2) Test clips or other wire to connect the power switch to the PowerBlock

Software

I started out using the awesome Combian 64, but ended up eventually rolling my own. While I could go into the Raspbian/Vice build I have made for this, I recommended starting with Combian 64 for ease of use.

You will need to install the PowerBlock driver for it to work.

Links

STL Files

Having only a native app with no web alternative

I read a book back in 2009 called the Big Switch, which proclaimed that the end of native apps (or as they were called in those days “computer programs”) was nigh, and in the future every app would reside on the web where it could be accessed by any internet-capable device regardless of OS.

Unfortunately for Windows, WebOS, and BlackBerry, that prediction didn’t quite come true. At least, not soon enough. Within two years of its publication, lazy yupsters were excreting mobile apps that did little more than access information from the internet, something that could easily be accomplished with a website. There was a time when you could not even view photos on Instagram without the mobile app, which means you could not view them at all on a computer.

The fact that so many new digital products were only available in native mobile form means that only those with the right device and OS could access them. The worth of an OS became almost entirely a function of how many apps were available. This led to a vicious cycle as also-ran platforms failed to attract developers as they would rather go where the users were… where the apps already were.

It’s hard to say if a bigger emphasis on the web would have saved every OS that was not iOS or Android, but even having a third option would be better than the stagnant duopoly we are left with today. For certain applications, natively installed software is still required or at least optimal. That might include programs that do (or should) not rely on data from the internet or nuanced interfaces that utilize a device’s sensors. But if your product is primarily information rather than functionality, there is no excuse not to put it on the web.

The future belongs to those who transcend the platform. It is good for the user, and it is good for the market. Already the industry is getting its comeuppance as “app fatigue” has set in, and users have stopped installing new apps, and started deleting them instead. I can’t say I didn’t predict this.

Bottom line

Wherever possible, design to be independent of platform.

In the wake of multiple gun-related atrocities in the past few months – Orlando, Las Vegas, and Parkland most recently – I wanted to try and offer a less-biased approach to the background of the “gun issue” in the United States.

From outside the US, I know it’s difficult to understand what the problem is. After all, we have more guns per capita than pretty much anyone else, and we also have a higher rate of gun violence. The global correlation numbers are pretty clear: if we had less guns, we’d have less gun-related violence. I’m not sure you need statistics for that, since it’s kind of like saying “if we had less pasta, we’d have less Italian-related food.” But it’s easy to simply ask, “why don’t you guys just cut back on the guns?”

To begin with, there’s the 2nd amendment to the US Constitution, one of our original 10 “Bill of Rights.” That’s not something we take lightly: the Constitution is literally the bedrock of our entire system of government and more than a little of our entire culture as Americans. It probably has more to do with the national identity of “American” than almost anything else; the Constitution is, for us, what takes a mob of immigrants and turns them into one nation. It’s difficult to explain to non-US citizens, because the Constitution is something we become emotionally connected to at a very young age.

A Brief History of the Second Amendment

There’s a lot of misinformation about Number Two’s origins and history. Some believe it was because British soldiers were confiscating weapons from Americans who needed them to hunt, in order to feed their families. Others believe it was because those soldiers wanted to prevent American colonial resistance. Both of those were likely things that happened, although in reading histories of the time I’m personally pressed to find a lot of evidence of the former. What we do have, however, are fairly copious notes from the original Constitutional Congress where the amendment was first presented.

James Madison proposed it originally, and it was seen as a way to provide more power to state militias. Keep in mind that the concept of Federalization was extremely unpopular with most states at the time, each of whom had previously operated more or less autonomously under British rule. Today’s “state militias” are, more or less, the state National Guard units nominally under the control of each state’s Governor.  Madison was fairly clear that the measure was intended to give states the power to fight back against a tyrannical Federal government. Even today, most gun regulation comes from the state level, not the Federal level, with a patchwork of laws and regulations spread across the current 50 states. For example, I believe it’s 44 states that have open-carry laws, even though nothing in the 2nd amendment has ever been interpreted as guaranteeing a right to open carry.

The Amendment reads:

A well regulated Militia, being necessary to the security of a free State, the right of the people to keep and bear Arms, shall not be infringed.

You might wonder, then, how much training we require for gun ownership, given that training would seem to be a basic tenet for a “well regulated Militia.” The answer is none. We largely can’t even agree on whether the amendment provides for a collective right – the right for states to arm its militias – or an individual right – the right for individuals to own guns. We’ve more or less opted for the latter, but it remains a hotly debated issue in scholarly works.

(I should note that we do indeed train our militia members quite extensively, but that has nothing to do with owning guns; not every National Guard member owns personal weapons.)

Our ultimate authority on the Constitution is our Supreme Court, which is tasked with interpreting the Constitution, and its amendments, for modern times. Back in 1876 and again in 1886, the Court held that gun ownership was not an individual right, and that the amendment only addressed Federal gun regulation, not state regulation. They held that same stance again in 1894.

We didn’t hear from the Court again until 1939, where some yahoo got arrested for carrying a banned sawed-off shotgun across state lines. This is the first time the Court acknowledged any Federal authority over guns, stating that, “in the absence of any evidence tending to show that possession or use of a ‘shotgun having a barrel of less than eighteen inches in length’ at this time has some reasonable relationship to the preservation or efficiency of a well regulated militia, we cannot say that the Second Amendment guarantees the right to keep and bear such an instrument.” This was also about the last time a “militia” played into the Court’s thinking.

Almost 70 years later, in 2008, the Court took on Number Two again. This is the first time the Court more or less discarded the “well regulated Militia” bit, stating in part that, “The Second Amendment protects an individual right to possess a firearm unconnected with service in a militia, and to use that arm for traditionally lawful purposes, such as self-defense within the home.” This was a big change, but it really didn’t reflect anything more than common thought at the time. By 2008, we had more guns in private ownership that citizens, and very, very, very few of those gun owners were in the militia.

And that’s more or less where we’re at from a legal perspective.

Gun Culture

Guns are scary devices, with good reason, and so terms like “Gun Culture” can evoke some negative feelings. It’s no different than “Car Culture,” though, which sounds much more West Coast-cool, and Gun and Car Cultures have a lot in common.

We don’t have a Constitutional right to keep and drive cars, but if anyone tried to regular their use more than we already do, or God forbid take them away, there’d be a hue and cry like you’d never believe. Cars are a legitimately vital part of many people’s lives, and they play a similar, fundamental role in the American identity as guns. Cars give us personal freedom. They’re one of the first big milestones a young American achieves as they grow into adults. Emotionally, for us, they’re a huge deal. And guns, for many Americans, fall into that same emotional/identity zone. That’s not meant to be a positive or negative statement; it simply “is.” Few Americans need guns to survive; few Americans belong to a militia (which would provide them with guns anyway, these days; the National Guard is not a BYOG club). But, like cars, they’re very much a part of our country’s basic weft and weave.

Making Changes

Notwithstanding former Supreme Court Justice Stevens’ comments recently, changing the Constitution isn’t easy. It’s purposefully difficult, for the same reasons you don’t just go down to the basement and start jackhammering your house’s foundations. The Constitution has far-ranging consequences that are difficult to grasp, and you don’t mess with it lightly. The only enumerated way to change the Constitution is to get two-thirds of both the Senate and the House to agree, and then three-quarters of the states to ratify the change. If this seems impossible, it’s definitely close, which is why we’ve only done it a handful of times in our 200+ year history.

And there’s no guarantee that even a full repeat of Two would matter. 44 states guarantee a right to bear terms in their Constitutions, and absent an explicit Federal override, those state laws would “win.” So we’d need to not only repeal Two, we’d need to add a new amendment to potentially override state laws. Not happening.

Which leads us to the very reasonable question of, “well, what changes could we make short of a Constitutional amendment?” Plenty. It’s questionable how much Federal regulation would be acceptable, given previous Supreme Court opinions, but states could do almost anything they wanted. And that’s likely the best place to regular almost anything in the US, believe it or not. We’re a country of more than 350,000,000 citizens. From a land mass and population perspective, we’re as big as Western Europe, and we have just as much cultural diversity. If you think having the big-bad EU issue Europe-spanning regulations is unpopular over there, well, the Federal government’s edicts are just about as loved over here.

Activists on all sides of any issue try to use the Federal government as a blunt instrument for two reasons. First, it’s perceived to be easier to pass one law on something that to pass fifty. Second, people get really bent out of shape when they see other people doing something they don’t like, even half a continent away. And so the Feds are seen as a way of enforcing one’s will on the masses. It goes over about as well as you might expect, given that the makeup of the US is more like eleven loosely-joined nations. I’m personally a big fan of state-based regulations, because it puts the power and the decision closer to the people it affects. I think the Federal government has its place helping us accomplish the Big Things we couldn’t do on our own, but I’m largely a states-rights fan. I’d personally be squeamish of major Federal regulations on almost anything, because those are so often heavy-handed, badly managed, and overwrought.

The Non-Debate

But here’s the real difficulty: conspiracy theorists (my phrase, which I understand is a bias on my part) believe that “the liberals” are coming to take our guns; given the resounding lack of factual evidence for that, it’s a theory I can’t even discuss. But basically, pro-gun anti-regulation advocates largely refuse to engage in debate.

After the Parkland massacre, Marco Rubio made a statement, which in part reads:

Protest is good way of making a point, but making a change will require both sides finding common ground

That’s true, and it’s the way any system of democracy is meant to work. The difficulty is that anti-regulation advocates simply refuse to engage in debate, discussion, or search for a common ground. For the most part, they simply insult, bully, and demean anyone taking any view less that total gun ownership freedom without restriction. It’s fine if your stance is that there should be no restrictions, but you should be able to state that, and mount a logical defense to your position that could be used in a debate on the issue.

Let’s be clear on something: there are millions of responsible, law-abiding, intelligent people in the US who own guns. I own several guns, and most of my friends do. I enjoy shooting targets at the range, and when I’m at my relatively isolated cabin I enjoy the extra peace of mind the shotgun offers against large, hungry predators. I, and most of my gun-owning friends, would be happy to engage in a discussion about how guns might be more intelligently regulated, and we do engage in those debates between ourselves. Some of my friends are members of organizations that advocate against gun regulation, and are a bit weary of those organizations’ hard-line stance against even having a discussion of the issues.

Most of us feel that for the purposes of discussion, anything can and should be on the table. Mandatory training. Bans on certain types of weapons. Limits on ammunition magazine sizes. Hell, someone mentioned mandatory militia service for gun owners, which is at least something we could talk about, you’d think. Some sides routinely fight against even simple things like gun registration, stating that making people register their guns is one step away from the Federal government knowing where all the guns are when it’s time to take them all back, which strikes me as a little paranoid and counterproductive (although I admit it’s a valid position in a debate of the issue). The politicians’ “thoughts and prayers” responses are driven largely by their loyalty to the refusal to even enter into a debate.

And let’s be clear on this, too: most conservative politicians aren’t even offering solutions. Not even bad ideas. The problem gets shoved off to “mental health” in a country that can’t even decide if it wants all of its citizens to even have health insurance, and that’s it. We can’t even get a crappy proposal to start discussing, which is why we can’t get any change.

Gun advocacy groups should have a valuable role to play in a national discussion on gun regulations. Unfortunately, they’re backing themselves into a corner. By refusing to even discuss the situation, and by focusing on a strategy of bullying to silence their opposition, they’re creating a situation where compromise can’t happen. Rather than bringing an opposing voice to the table, pro-gun organizations could eventually find themselves excluded from the table entirely.

The fact that we can’t even get a meaningful, civil debate off the ground in the US is simply inexcusable. That alone represents a bigger failing of basic democracy than the actual underlying political issues.

And Here We Sit

And so that’s more or less the current situation. One “side” is pushing for stronger regulations on at least certain kinds of guns, accessories, and ammunition; the other “side” is calling them names and basically refusing to talk about it. In the middle sits the vast, vast, vast majority of Americans, most of whom wouldn’t find anything at all toxic about a discussion. 

Notably uninvited from discussing the issue are the men and women of our militias, who would presumably have some thoughts about the 2nd amendment that ensures they can be armed. Largely uninvited are the vast numbers of men and women in law enforcement, who presumably have some thoughts about gun ownership (many of them own guns privately as well as for work), enforcement, and safety. We’ve yet to hear anything substantive from them except, perhaps, in our own social media channels.

It’d be nice to say that the debate rages on, but it doesn’t. There is no debate, and it’s that singular failing of the entire concept of democracy that’s the most saddening to me personally. Whatever your stance on the issue, I think we as a people should hear it. We should all be able to put our concerns, fears, and desires on the table, and find someplace in the middle where we hit the best possible balance between it all – and then try that for a minute.

I’ve tried not to share my personal stance on the matter of gun regulation, because right now that’s not what should matter. What should matter is that we all get a fair shot to have our opinions represented and heard, and that we can’t is the ultimate problem. I hope that’s something we can get fixed, and soon.

Knowing what a politically fraught topic this is, I hope I’ve managed to keep this fairly neutral on the issue itself, if not around my disrespect for advocacy organizations’ unwillingness to engage on the actual issues. I want to acknowledge that all “sides” of this debate (there are more than two, in my feeling) have made bad calls along the way, but hold firm to my feeling that the one thing we should all agree on is that an actual honest discussion can’t possibly hurt. If you’ve got something to add, please do sound off in the comments.

Please try to avoid pejorative terms like “right-wingers” or “libs” and so on; even when we disagree, we’re all meant to be one nation. We have to acknowledge that when we’re discussing something so embedded in our national fabric, it’s going to be tense. Try to make it not tense.

Discussion Guide

I’ll offer this, for what little it’s worth. Whenever you’ve a tense issue like this, with tempers running hot on all sides, try to set your own temper aside and create some quiet space. For example, just ask, without judging or arguing, you opponent a few simple questions:

• What are your concerns? What are you trying to fix?
• What is your proposed solution?

If you can do that without name-calling, and get some solid, quiet answers, you can move on.

• Let me tell you what fears your proposal raises in me. You don’t need to agree, but I’d like it if you could just hear me out.
• If we changed __ about your solution, it would help alleviate some of my fears. How would that change it for you, though?
• What if we added ___ – would that help alleviate your remaining concerns to at least some degree?

Try and aim for the smallest possible point of agreement. We don’t need to sole any entire problem in one fell swoop; we can pick away at it until we solve the bits we can solve together. We won’t be able to solve it all, usually, and that’s fine. As tempers and situations change, we can continue to pick away it it.

If we can discuss it.