TASK \ OSAIXA/UXDG/UXFreeBSDHP-UXIRIXLinuxMac OS XNCR UnixNetBSDOpenBSDReliantSCO OpenServerSolarisSunOS 4Tru64UltrixUNICOSOS / TASKOS notesBy IBM, with input from System V, BSD, etc. Runs mainly on IBM RS/6000, System p, Power Systems, and related hardware.Apple 1988-1995. Based on AT&T SysV.2.2 with extensions from V.3, V.4, and BSD 4.2 and 4.3Data General was aquired by EMC in 1999.Derived from 4.4BSD-Lite and 386BSD. This table does not include external packages from /usr/ports.Hewlett-PackardSGI. SVR4-based

(rh) = Red Hat,
Mandrake, SUSE,...
(deb) = Debian, Libranet,...
(fed) = Fedora
(gen) = Gentoo
(md) = Mandrake/Mandriva
(SUSE) = SUSE

Darwin is the Open Source foundation for Mac OS X, descended from NeXTStepaka MP-RASDerived from 4.4BSD-Lite and 386BSD. This table does not include external packages from /usr/pkg.Forked from NetBSD in 1995. Security emphasis. This table does not include external packages from /usr/ports.Fujitsu Siemens. Originally called SINIX. Based on DC/OSX from Pyramid, SINIX
from Siemens, and SVR4. To be replaced with Solaris by 2008.SCO UnixWare 7.0.1& SCO UnixWare 7.1.1 became Caldera Open Unix and later became SCO again. SVR4-based.

Solaris 2.0-2.6, 7, 8, 9, 10, 11
OpenSolaris (SunOS 5.*)
SVR4-based.
An open-source version can
be found at illumos.org

Solaris 1.*
BSD-based (Digital Unix, OSF/1)
(4)= 4.0F/G
(5)= 5.0AAn early DEC Unix, superceded by Digital Unix (now Tru64).Cray Inc., formed from the March 2000 merger of Tera Computer Company and Cray ResearchOS notesadministrative GUIsmit
smitty
wsm Findersysadm

sysinstall

bsdconfig (10+)

sam

smh (11.31+)

 /usr/sysadm/bin/sysmgr (6.3+)
toolchest
/usr/Cadmin/bin/* (rh, FC2+, RHEL4+) system-config*
(rh) redhat-config*
linuxconf (obsolete)
(SUSE) yast2
(deb) dpkg-reconfigure
(md) drakconf

NetInfo Manager

sysadmsushi (obs)?

sysadm
WebSysAdmin

scoadmin

solstice
admintool
smc (8 01/01+)

?checklist
(5) sysman?checklist
/etc/install/install
xmenu administrative GUImanaging usersuseradd (5+)
userdel (5+)
usermod (5+)
lsuser
mkuser
chuser
rmuser
usrck
Finder
/etc/adduser
edit /etc/passwd useradd
userdel

adduser
chpass
rmuser
pw
vipw
bsdconfig (10+)

useradd
userdel
usermod
getprpw
modprpw
pwget

/usr/sysadm/privbin/
addUserAccount

passmgmt

/usr/Cadmin/bin/cpeople

useradd
usermod
userdel
adduser
chage
getent

Aqua: System
Preferences / Accounts
niutil (OS X Server)
dscluseradd
userdel
usermod
sysadm useradd
userdel
userinfo
usermod
user
vipwuseradd
userdel
adduser (interactive)
usermod
usersysadm
useradd
userdel scoadmin
useradd
userdel
useradd
userdel
usermod
getent
logins
/usr/sadm/bin/smuser (9+)
groupaddedit /etc/passwd and create home directory

useradd
adduser
userdel
removeuser
dxaccounts

adduser
removeuser xadmin
udbgen
udbrstrict
nu scriptsmanaging usersTASK \ OS AIXA/UXDG/UXFreeBSDHP-UXIRIXLinux
Mac OS XNCR UnixNetBSDOpenBSDReliantSCO OpenServer
SolarisSunOS 4Tru64
UltrixUNICOSOS / TASK list hardware configuration

prtconf
lscfg
lsattr
lsdev
lspath

?dg_sysreport -g hardware

uname
sysctl -a
dmesg
cat /var/run/dmesg.boot
pciconf -l
usbconfig
camcontrol devlist
devinfo

ioscan
dmesg (if you're lucky)
adb
model
stm (from the SupportPlus CDROM)
bdf (like most vendors' df)
getconf
machinfo (11.23+)
print_manifest

hinv

sysinfo -vv

arch
uname
dmesg (if you're lucky)
cat /var/log/dmesg
/proc/*
lshw
dmidecode
lspci
lspnp
lsscsi
lsusb
lsmod
(SUSE) hwinfo
/sys/devices/*

hostinfo
ioreg -bls
system_profiler
prtconf
devstat
machinetype
more /etc/.osmdmesg
more
/var/run/dmesg.boot
cat /kern/msgbufsysctl hw
dmesg
cat /var/run/dmesg.boot
pcidump
acpidump

autoconf -l
showconf
uname -p
uname -M
mpcntl -I

hwconfig
hw
scoadmin/DCU

/usr/lib/X11/testtools/
z35SysInfo

prtconf -v
/usr/platform/`uname -i`/
sbin/prtdiag -v (sun4u and sun4d only)
arch -k
psrinfo -v
isainfo -v
dmesg
iostat -En
prtfru
cfgadm -l
/etc/path_to_inst
smbios

dmesg (if you're lucky) 

dia -R full
vmstat -P
hwmgr -get category
(5) hwmgr -show
devices
psrinfo
pset_info
sys_check
/usr/bin/X11/dxsysinfo

dmesg (if you're lucky)

machid
getconf
uname -a

list hardware configuration unique id useful for licensinguname -u
uname -m
lsattr -El sys0 -a systemid??cat /etc/hostid
sysctl kern.hostuuid uname -i ?/sbin/ifconfig eth0

system_profiler SPHardwareDataType |
grep "Serial Number:" |
awk '{print $4}'

?????hostidhostidconsvar -g sys_serial_num??unique id useful for licensingshow/set
EEPROM/NVRAM
valuesbootlist -o -m normal
n/adg_sysctl |
grep BOOTPATHeeprom (SPARC only)setboot
stmnvram
printenv/setenv (at command monitor)/dev/nvram (if you have it)
hwclock
nvsetenv nvram?eepromeepromin bdm
(at boot stop
or ctrl-a)?eepromeeprom(4.0D+) consvar??show/set
EEPROM/NVRAM
valuesadd device without rebootcfgmgr -v
mkdev?N/Acamcontrol
atacontrol
kldload
/etc/rc.d/devfs restart

inf -e
ioscan -fnC devtype;
insf -C devtype

scsiha -pr
ioconfig -f /hw
scsiadminswap
-b bus -p
udev (2.6+)
udevtrigger
modprobe
kerneld
insmod
hotplug
cardctl
(automatic — autdodiskmounter)devmgmt
mktable scsictl
atactl (auto-detected)reinitdev
changedev mkdev

(10+) add_drv
devfsadm. pre-Solaris 7 HW 11/99, use:
drvconfig; devlinks; {disks, tapes, ports}

/usr/lib/symsm/bin/hot_add (RDAC)
/etc/symsm/rmparams (RDAC)

/dev/MKDEV

(3.2.*) scu scan edt
(4.0) scu scan edt; disklabel -rw rzXX label (for disks)
(5.x) dsfmgr -k; disklabel -rw dskn auto
hwmgr -scan scsi

/dev/MAKEDEV?add device without rebootremove device rmdev??camcontrol stop
camcontrol eject rmsf????????

(10+) rem_drv
devfsadm

????remove devicetape device/dev/rmt0/dev/rdsk/c8d?s0/dev/rmt/0/dev/rsa0
/dev/*wt*
/dev/*sa[0-9]*/dev/rmt/0m/dev/tape
/dev/nrtape /dev/st0?/dev/rmt/c*/dev/st0/dev/[e][n][r]st[0-9]*/dev/ios0/rstape001
autoconf -1 |
grep TAPE/dev/rct0/dev/rmt/0/dev/rst0(5.x) /dev/ntape /dev/tape/dev/nrmtXh where n=non-rewinding, X a number, h=densitytpmnttape devicestdin/ stdout/ stderr???/dev/fd/[012]?/dev/fd/[012]/dev/fd/[012]/dev/fd/[012]?/dev/fd/[012]/dev/stdin
/dev/stdout
/dev/stderr
/dev/fd/[012]??/dev/fd/[012]????stdin/ stdout/ stderrX kvm config???Xorg -configure
xf86configitemap/usr/bin/X11/xsetmon
/usr/gfx/setmon

xorgcfg
xorgconfig
Xorg -configure
system-config-display
xf86config
XFree86 -configure
redhat-config-xfree86
/etc/X11/?dm

? xf86config
xf86cfg
XFree86 -configureX -configure?hwconfigkdmconfig (x86)
fbconfig; m64config; (etc.)????X kvm configTASK \ OS AIXA/UXDG/UXFreeBSDHP-UXIRIXLinux
Mac OS XNCR UnixNetBSDOpenBSDReliantSCO OpenServer
SolarisSunOS 4Tru64
UltrixUNICOSOS / TASK read a disk label lspv?admpdisk -o list
admvdisk -o listdisklabel -r
gpart show (7.0+)

scsimgr (11.31+)
diskinfo
pvdisplay

prtvtocfdisk -lpdisk rawDev -dump
Disk Utilityprtvtocdisklabel -r
disk
mbrlabel
fdisk disklabel diskNamedkpart -lfsnameprtvtocdkinfodisklabel -rchpt -q?read a disk label whole disk in partition hdisk[0,1,2,...]??c ? 10/dev/hda (e.g. if /dev/hda1
is a partition)(non-root partitions are mounted under /Volumes)s0c or d
sysctl kern.rawpartition c /dev/ios0/rsdisk???s7
(???=000-999) /dev/1s# (#=0,1,...)2cccs7whole disk in partition label a disk mkvg?admpdiskfdisk
disklabel -wr
sysinstall
gpart add (7+)
sade (6.3+)
bsdconfig (10+)pvcreate

dvhtool
fx -x

cfdisk
fdisk
gdisk
e2label diskutil

sysadm
fdisk
edvtoc
prtvtoc
mkpart
updatescsi
labelit

disklabel -wr
mbrlabel disklabel -E diskNamedksetupdivvyformat
prtvtoc
(x86) fdisk format disklabel -rwchpt -a/etc/labelitlabel a disk partition a disk mklvdp
Finder: Apple HD SC Setup admpdisk fdisk
sysinstall
gpart add (7.0+)
sade (6.3+)
bsdconfig (10+)smh (11.31+)
lvcreate
sam fxparted (if you have it)
cfdisk
fdisk
gdisk
pdisk (on a MAC)
(deb) mac-fdisk (on a MAC)
(md) diskdrakepdiskfdiskdisklabel -ifdisk -e diskNamedkpart -edivvyformat
fmthardformatdisklabel -echpt -p/etc/install/installpartition a disk TASK \ OS AIXA/UXDG/UXFreeBSDHP-UXIRIXLinux
Mac OS XNCR UnixNetBSDOpenBSDReliantSCO OpenServer
SolarisSunOS 4Tru64
UltrixUNICOSOS / TASK kernel/unix/unix/dgux /boot/kernel/kernel/hpux (9)
/stand/vmunix (10+)/unix

/boot/vmlinuz*
/boot/bootlx

(see /etc/lilo.conf or /boot/grub/menu.lst)

/mach_kernel/unix
/stand/unix/netbsd/bsd

/unix

/stand/unix

/unix/kernel/genunix
/platform/`uname -m`/
 kernel/unix
/platform/`uname -m`/
 kernel/sparcv9/unix (7+) /vmunix/vmunix/vmunix/unicoskernelshow/set kernel parametersvmo
ioo
schedo
/etc/kconfig
Findervi /var/Build/
system.hostname;;
reboot

/boot/loader.conf
sysctl
/etc/sysctl.conf
kenv

smh (11.31+)
sam
sysdef
kmtune (11+)
kctune (11.23+)
getconfsystune/proc/*
/proc/sys/*
sysctl
/etc/sysctl.conf /usr/sbin/sysctl
launchctl limit
/etc/launchd.confsysdef
getconf
/etc/conf/cf.d/
quicktune.cfg sysctl
/etc/sysctl.conf sysctl
/etc/sysctl.confidtune
cat /etc/opt/cf.d/stune
cat /etc/opt/cf.d/mtune
sysdef -i

/etc/conf/cf.d/configure -x

sysdef

sysdef
getconf
cat /etc/system
ndd
adb -k

adb

/usr/kvm/sys/
`arch -k`/conf

dxkerneltuner
sysconfig adb
kdb?show/set kernel parameterslimit physical memory rmss??hw.physmem (in /boot/loader.conf)??mem= (in lilo.conf or /boot/grub/grub.conf) nvram bootargs="maxmem=MB"??machine memory (at the "boot" prompt) ??/etc/project
zonecfg????limit physical memory loaded kernel modules genkex
vxketadm -s ??kldstatkmadmin -s
kcmodule -a (11.23+)?lsmodkextstat?modstatmodstat?modadminmodinfo????loaded kernel modules load module vxketadm modname load
cfgmgr
mkdev -l devname??kldloadkmadmin -L
kcmodule module=loaded (11.23+)?modprobe
insmodkextload?modloadmodload?modadminmodload????load module unload module rmdev -l devname??kldunloadkmadmin -U
kcmodule module=unused (11.23+)?rmmod
modprobe -r kextunload?modunloadmodunload?modadminmodunload????unload module make disk bootablebosboot -a?dg_sysctlboot0cfg
disklabel
gpart bootcode (7+)
sade (6.3+)mkbootdvhtool

(rh) mkbootdisk
(deb) fdisk -A
(and lilo to manipulate mbr)

blessdklayoutfdisk -i
installboot /usr/mdec/installboot⁵
/usr/mdec/biosboot⁵dksetup?installboot /usr/platform/
`uname -i`
/lib/fs/ufs/bootblk raw_device_file/usr/kvm/mdec
installboot [-hltv]
protobootblk
bootdevice (4.x+) disklabel -rw -t [ufs|advfs]??make disk bootablestartup scripts /etc/rc*
/etc/inittab?/etc/rc*/etc/rc* (4-)
/etc/rc.d/* (5+)
/usr/local/etc/rc.d/*

/etc/rc* (9)

/etc/rc.config.d (preferred)
/sbin/rc* (10+)
/sbin/init.d/

/etc/rc*
/etc/init.d/ /etc/rc* (but may vary)
/etc/init.d/
chkconfig
service/System/Library/
StartupItems/*,
/Library/StartupItems/,
/etc/rc*

/etc/rc*
/etc/init.d

 
/etc/rc
/etc/rc.*
/etc/rc.d/*  
/etc/rc*
/etc/netstart/etc/rc*/etc/rc*/etc/rc*
/etc/init.d/
(10+) svcadm
(10+) svcs/etc/rc*/sbin/init.d; links in /sbin/rc?.d/etc/rc*?startup scripts start/ stop/ config servicesstartsrc
stopsrc
lssrc
refresh
srcmstr??service (8.1+)?chkconfig(rh) service
(rh) chkconfig
(deb) sysv-rc-conflaunchctl?????svcs (10+)
svcadm (10+)????start/ stop/ config servicesshutdown (& power off if possible) shutdown -Fh /etc/shutdown -h
Finder?shutdown -p now
poweroff
halt -pshutdown -yh 0 /etc/shutdown -y -g0 -i0 -p shutdown -Ph now
shutdown -y -g0 -i0
halt
poweroffhalt?shutdown -p now shutdown -hp now ?shutdown -i0 shutdown -y -g0 -i5 ????shutdown (& power off if possible)run levels ¹
*=normal states
for more detail
see
www.phildev.net/runlevels.html2: multiuser
6: reboot??

0: halt and power off
1: single-user mode
6: reboot
c: block further logins
q: rescan ttys(5) file

kernel security levels:
-1*: permanently insecure
0: insecure
1: secure
2: highly secure
3: network secure

0: shut down
s,S: single-user
1: sys admin
2: multiuser
3*: share NFS
4*: start GUI
5: user-defined
6: user-defined0: power-down
s,S: single-user
1: sys admin
2: multiuser (no NFS)
3*: multiuser
4: user-defined
5: firmware monitor(set in /etc/inittab)
0: halt
s,S,1: vendor-dependent
1: single-user
2-5*: multiuser
6: reboot?(set in /etc/inittab)
0: shutdown
s,S: single-user
1: sys admin
2*: multi-user
3: share NFS
4: user-defined
5: firmware
6: reboot
-1: permanently insecure
0*: insecure
1*: secure
2: highly secure-1: permanently
insecure
0*: insecure
1*: secure
2: highly secure0: power-down
s,S: single-user
1: sys admin
2*: multiuser
3+4*: user-defined
5: firmware update
6: reboot?

0: firmware monitor
s,S: single-user
1: sys admin
2: multiuser
3*: share NFS
4*: user-defined
5: power-down if possible
6: reboot

?0: shut down
1: single user
2: multi-user, no NFS
3: multi-user
6: reboot0: shut down
1: single user
2: multi-user, no NFS
3: multi-user
6: reboot?

run levels ¹
*=normal states
for more detail
see
www.phildev.net/runlevels.html

show runlevel ¹who -r who -r sysctl kern.securelevelwho -rwho -r/sbin/runlevel(runlevels unused)who -rsysctl kern.securelevelsysctl kern.securelevelwho -rwho -rwho -r?who -r?who -rshow runlevel ¹time zone info/etc/environment ($TZ)/etc/TIMEZONE/etc/TIMEZONE/etc/localtime
/usr/share/zoneinfo /usr/lib/tztab/usr/lib/locale/TZ/
/etc/TIMEZONE /usr/share/zoneinfo/
/etc/localtime/usr/share/zoneinfo//usr/lib/locale/TZ/etc/localtime
/usr/share/zoneinfo//usr/share/zoneinfo/
/etc/localtime ??/usr/share/lib/zoneinfo/?/etc/zoneinfo??time zone infocheck swap space lsps -a/etc/swap -l admswap -o list -buv
dg_sysreport -p freeswap
systat -swapswapinfo
pstat -sswapinfoswap -s
swap -lswapon -s
cat /proc/meminfo
cat /proc/swaps
free ls -l /var/vm
vm_stat swap -s
swap -l
sar -rswapctl -l
pstat -s
systat swap swapctlswap -s
swap -lswap -lswap -s
swap -lpstat -sswapon -s?/etc/swappercheck swap space core dump managementchcore
sysdumpdev  kern.sugid_coredump
kern.capmode_coredump
kern.coredump
kern.nodump_coredump
kern.corefilecoreadm /proc/sys/kernel/core_pattern
/proc/pid/coredump_filter      coreadm    core dump managementbind process to CPU bindprocessor??cpuset
rctl (9+)psrset -b
mpsched -c
intctlrunontaskset (sched-utils) ??????pbind?runon
pset_create
pset_info
psradm
??bind process to CPU TASK \ OS AIXA/UXDG/UXFreeBSDHP-UXIRIXLinux
Mac OS XNCR UnixNetBSDOpenBSDReliantSCO OpenServer
SolarisSunOS 4Tru64
UltrixUNICOSOS / TASK "normal" filesystem jfsufsdg/uxufs/ffs/ufs2 (5+)
ext2
zfs (7+)

hfs
vxfs (11.0+)

efs, xfsext2
ext3
ReiserFS ufs
hfs+ vxfsffs (was ufs)ffs (was ufs)vxfs
ufs (root)htfs (OpenServer 5)
vxfsufs
zfs (OpenSolaris)4.2

(4) ufs
(5) AdvFS

ufsNC1FS
C2FS "normal" filesystem volume-based filesystem jfs2?admvdisk

ccd
vinum
geom (5+)
zfs (7+)

jfs (vxfs)
LVM
VxVM ($)xlv, xvm, xfsLVM, LVM2
EVMS
VxVM ($) ?vdisk
VxVM
ccd, raidframe
ccd
raid
softraidvdiskvdiskzfs(10+)
Solstice DiskSuite
VxVM ($)? Advfs
LSM N/A?volume-based filesystem file system
description/etc/filesystems
mount
lsfs/etc/fstab/etc/fstab/etc/fstab
zfs (7+)/etc/checklist
(9) 
/etc/fstab
(10+)/etc/fstab /etc/fstabNetInfo/etc/vfstab/etc/fstab /etc/fstab /etc/vfstab
/etc/dktab

/etc/mnttab
/etc/default/filesys
/etc/vfstab (local)
/etc/rmtab (remote)

/etc/vfstab (local)/etc/fstab /etc/fstab
AdvFS: /etc/fdmns /etc/fstab /etc/fstabfile system
descriptionvolume manipulation

smitty vg
smitty lvm
smitty jfs
smitty storage
lsvg; varyonvg; varyoffvg
mkvg; chvg;
reducevg; extendvg
exportvg; importvg
mklv; chlv; rmlv
mkfs; crfs; rmfs

?sysadm

vinum
ccdconfig
geom (5.3+)
gstripe (5.3+)
gmirror (5.3+)
graid3 (5.3+)
zpool

vg*; lv*; pv*
extendfs
fsadm

xlv_mgr
xvm

e2fsadm
lvcreate
lvremove
lvextend
vgdisplayDisk Utility
pdisk
diskutil

vxva

dkconfig

ccdconfig
raidctlccdconfig
raidctl
bioctldkconfig
dkmirror
dkmap ?

zpool (10+)

meta*
(Solstice DiskSuite;
Solaris Volume Mgr (9+))
(7-, found in
/usr/opt/SUNWmd/
 sbin/)

/opt/VRTSvxva/bin/vxva
(Veritas Volume Mgr) ($)

?AdvFS: mkfdmn, mkfset, addvol, showfdmn
LSM: volassist or volmake, volume, volplex and volsd, volprint?/etc/install/installvolume manipulationcreate filesystemcrfs
mkfs/etc/newfs
/etc/mkfsadmfilesystemnewfs
zfs (7+)
newfs_msdos  mkfs
newfs mkfsmke2fs
mkreiserfs
mkdosfs
mkfs.*newfs
newfs_hfs
newfs_mdosmkfsnewfs
makefs newfsmkfs
newfs divvy (OpenServer 5)
mkfsnewfs
mkfs
zfs (10u2+)newfs
mkfs AdvFS: mkfdmn
UFS: newfs
MFS: mfsnewfs
mkfs bb
mkfs
labelitcreate filesystemfile system debugging and recoverydfsck
fsck
fsdb fsckfsck
fsdb fsck
fsdb
clri fsck
fsdbfsck
fsdb
clri
xfs_repair
fsck
debugfs
e2undel fsck
fsdb
clri ?fsck
fsdb
clrifsck
fsdb
clri
scan_ffs?fsck
fsdbfsck
fsdb
clri????file system debugging and recoverycreate non-0-length empty filelmktemp??truncate -s size[K|M|G] filename
dd if=/dev/zero of=filename bs=size[b|k|m|g] \
count=desiredpreallocmkfile

fallocate

dd if=/dev/zero of=filename
bs=1024k count=desired

mkfile
dd if=/dev/zero of=filename bs=1024 count=desireddd if=/dev/zero
of=filename bs=1024
count=desireddd if=/dev/zero of=filename bs=1m count=desired

dd if=/dev/zero
of=filename bs=1024
count=desired

dd if=/dev/zero of=filename bs=1 count=1 seek=desired-1
(sparse file)

mkfile
dd if=/dev/zero of=filename bs=1kdd if=/dev/zero of=filenmae bs=1024 count=desiredmkfilemkfiledd if=/dev/zero of=filename bs=1024k count=desired??create non-0-length empty fileconvert DOS-format text to Unix formatstrings dosfile.txt> unixfile.txt??tr -d '\r' < input> outputdos2ux?dos2unix??????dos2unix????convert DOS-format text to Unix formatmount CDROM

mount -v cdrfs -o ro
/dev/cd0 /cdrom

smitty cdrom

Findermount -t cdrom
/dev/pdsk/disk/cdrom

mount /cdrom
mount_cd9660

mount -t cd9960 /dev/cd0 mountpoint

pfs_mountd
pfsd
pfs_mount
mount /dev/dsk/cxtydz⁴
/SD_CDROM

mount -orr /dev/cdrom /mnt/cdrom (requires "Rock Ridge extension" patches, 11+)

(see man mediad)

(mounts automatically when CD inserted)

mount /mnt/cdrom
(deb) mount /cdrom

(automatic — automounter)

diskutil mount /dev/disk1s0

mount -F cdfs
/dev/cdrom/cdromX
/cdrommount -t cd9660 -o ro /dev/cd0a pathmount /dev/cd0a pathmount -F hs ...
sysadm mount /dev/cd0 path/etc/init.d/vold start; volcheck or
mount -F hsfs -o ro /dev/dsk/c0t6d0s2 /cdrommount -t hsfs -r
/dev/sr0 /cdrom

mount /dev/disk/cdromXa
/mnt
mount -t cdfs -o noversion /dev/disk/cdrom3c /cdrom

??mount CDROMeject CDROM ?(manually)?camcontrol eject devID
cdcontrol eject [-f device] ?ejectejectdiskutil eject /dev/disk1s0
/Volumes/theCDmountpoint?eject cdrom
eject cd0
eject -n cdio eject
eject cd0
eject /dev/rcd0c ??eject cdrom ????eject CDROM create/mount ISO imagemkcd??mkisofs
mdconfig; mountnohup pfs_mountd&;
nohup pfsd&;
pfs_mount -t iso9660 -x unix pathToIso mountPoint mkisofs (freeware)

mkisofs

mount -o loop pathToIso
mountPoint

hdiutil?makefs -t cd9660
image_filedirectory;
vnconfig -c vnd0
image_file;
mount -t
cd9660 /dev/vnd0a mountPoint

mkhybrid -o image.iso dir

vnconfig vnd0 image.iso && mount -t cd9660 /dev/vnd0c mountPoint

??

mkisofs²;DEVICE=`lofiadm -a /absolute_pathname/image.iso` ; mount -F hsfs -o ro
$DEVICE

????create/mount ISO imageACL managementaclget
aclput
acledit
ls -e??getfacl
setfacl
{get,set}{f,p}maclsacl
chacl
getacl
setacl

chacl

(see man acl)

getfacl
setfacl chmod
fsaclctl (-10.6) ?n/a???getfacl
setfacl ????ACL managementFibre Channel / SAN fcstat
lscfg -vl fcs0
lsattr -El fcs0
lspath??mptutil (8.0+)
showmount
gmultipathscsimgr (11.31+)
fcmsutil
fcdlist
tdutil
tdlist
tddiagcmgr
cxfs-config
cxfsmgr
cxfs_info

systool

multipath

??(treat as SCSI device) ???cfgadm
luxadm
stmsboot
mpathadm
fcinfo????Fibre Channel / SANTASK \ OS AIXA/UXDG/UXFreeBSDHP-UXIRIXLinux
Mac OS XNCR UnixNetBSDOpenBSDReliantSCO OpenServer
SolarisSunOS 4Tru64
UltrixUNICOSOS / TASK NFS share definitions/etc/exports
/etc/xtab /etc/exports/etc/exports/etc/exports/etc/exports
/etc/dfs/dfstab (11.31+)/etc/exports/etc/exportsNetInfo: /exports
Netinfo: /config/SharePoints/etc/dfs/dfstab
dfshares/etc/exports/etc/exports/etc/dfs/dfstab
/etc/dfs/sharetab/etc/exports/etc/dfs/dfstab
dfshares /etc/exports /etc/exports?/etc/exportsNFS share definitionsNFS share commandexportfs -a
mknfsexp/usr/etc/exportfs -a exportfs -a

kill -s HUP `cat /var/run/mountd.pid`

zfs set sharenfs="..." zfsdataset
service nfsd start

exportfs -a
share (11.31+)
shareall (11.31+)
share_nfs (11.31+)exportfs -a/etc/init.d/nfs-server reload
(rh) exportfs -akill -s HUP `cat /var/run/mountd.pid`share
shareall kill -s HUP `cat /var/run/mountd.pid`
/etc/rc.d/mountd reload kill -HUP `cat /var/run/mountd.pid`share
shareallexportfs -a
/etc/nfs startshare
shareallexportfs -a /sbin/init.d/nfs start; /sbin/init.d/nfsmount start?exportfs -aNFS share commandNFS informationnfsstat
nfso -as /usr/etc/nfsstatnfsstatnfsstatnfsstatmount | grep nfscat /proc/mountsnfsstat?nfsstatnfsstat?showmount
nfsstat

showmount
nfsstat

????NFS informationname resolution order/etc/netsvc.conf
/etc/resolv.conf?/etc/netsvc.conf/etc/host.conf (4-)
/etc/nsswitch.conf (5+)/etc/nsswitch.conf6+: /etc/nsswitch.conf
5.3: /etc/resolv.conf/etc/nsswitch.conf
/etc/resolv.confNetInfotcpconfig
/etc/netconfig /etc/nsswitch.conf
/etc/resolv.conf (obsolete)/etc/resolv.conf/etc/netconfig/etc/resolv.conf/etc/nsswitch.conf
getent (hardcoded
into libc)

/etc/nsswitch.conf
/etc/svc.conf (obsolete)

?/etc/hosts.usenamedname resolution ordershow network interface infoifconfig -a
no -a
netstat -i
netstat -v
lsattr -E -l entX
entstatnetstat -i
/etc/ifconfig -a netstat -i
ifconfig ifconfig -anwmgr (11.31+)
lanadmin
landiag
lanscan
netstat -in
ifconfig lanx/usr/etc/ifconfig -a
netstat -iaifconfig
ethtool ifconfig -a
netstat -a ifconfig -a
ildconfig
netstat -inifconfig -a
netstat -inifconfig -Aifconfig -a
inetstat -in
etherstat
cmxinfo

ifconfig -a
netstat -in
Network Config. Mgr. (GUI)

dladm
ndd
ifconfig -a
netstat -in
dladm
(8+) kstat -n hme0 | egrep 'ifspeed|duplex' ifconfig -a

ifconfig -a
netstat -in
hwmgr get attribute
-category network

ifconfig
netstat -iifconfig -a
netstat -inshow network interface infochange IP

smitty chinet

chdev -a netaddr=
newIP -I en#

mktcpip
rmtcpip

ifconfig
edit /etc/NETADDRS sysadm

edit /etc/rc.conf (option ifconfig_interfacename)

ifconfig

bsdconfig (10+)

/sbin/set_parms initial
ifconfig edit: /etc/{hosts,
sys_id,
resolv.conf,
TIMEZONE},
/etc/config/
{static-route.options,
ifconfig-1.options};
reboot

ifconfig
(rh) redhat-config-network
(fedora) system-config-network
(rh) edit: /etc/hosts,
/etc/sysconfig/network,
/etc/sysconfig/
network-scripts/
ifcfg-eth0
(deb) edit
/etc/network/interfaces;
/etc/init.d/networking
restart

System Preferences: Network
NetInfoifconfig
tcpconfig

edit /etc/rc.conf (ifconfig*)

edit /etc/ifconfig.interface

ifconfig

edit /etc/hostname.driver

sysadm
ifconfig (temp.)
vi /etc/default/inet

ifconfig

netconfig

edit: /etc/{resolv.conf, hosts, hostname.*, netmasks, nodename,
dumpadm.conf, inet/ipnodes, net/*/hosts}

sys-unconfig; reboot (this is overkill for a simple IP or hostname change)

edit /etc/hosts
/etc/hostname.* (4.x) netsetup
(5.x) sysman net_wizard or
netconfig?

netconfig
ifconfig
initif

/etc/install/install

change IPstart DHCP clientdhcpcdn/adhcpd

dhclient

service dhclient start interface

?proclaimdhcpcd
dhclient??dhclient

dhclient interface
(runtime)

add dhcp to
/etc/hostname.driver
(boot time)

??ifconfig interface dhcp?dhcpconf interface start ??start DHCP clientping one packet  ping -c 1 hostname  /usr/etc/ping hostname packetsize 1 ping -s -c 1 hostname ping -c 1 hostname  ping hostname 10 1 (9)
ping hostname -n 1 (10+)/usr/etc/ping -c 1 hostname  ping -c 1 hostnameping -c 1 hostnameping -c 1 hostname ping -c 1 hostname ping -c 1 hostnameping -c 1 hostnameping -c 1 hostnameping hostname  packetsize 1ping hostname
packetsize 1ping -c 1
 ping hostname
packetsize 1ping hostnameping one packet sniff networkiptrace
ipreport?tcpdumptcpdump

nettl
netfmt

snoop
tcpdump etherfind
tcpdump
wireshark (formerly ethereal)
etherapetcpdumptcpdump
tcpdumptcpdumptcpdump?snoopetherfindtcpdump??sniff networkroute definitions netstat -r
route
netstat -r sysadmedit /etc/rc.conf (option defaultrouter)
netstat -r
route
bsdconfig (10+)/etc/rc.config.d/netconf
netstat -r /etc/config/
static-route.optionsroute
(rh) /etc/sysconfig/network
(rh) /etc/sysconfig/static-routes
(deb) /etc/init.d/network
(deb) /etc/networknetstat -rtcpconfig
/etc/inet/defaultrouter
/etc/inet/gate*
netstat -r

netstat -r
route show
/etc/rc.conf
/etc/mygate

netstat -r
route show
/etc/mygate

netstat -rnetstat -r

/etc/defaultrouter
/etc/notrouter
/etc/gateways
in.routed
netstat -r
route add

/etc/defaultrouter
(kernel tweak or
adb to diable
forwarding) /etc/routes ? netstat -rroute definitions telnetd, ftpd banner/etc/security/login.cfg/etc/issue?/etc/ftpwelcome/etc/inetd.conf (telnet)/etc/issue/etc/issue.net (telnet)
(ftp varies; can use tcp wrappers)/etc/ftpwelcome/etc/issue
~ftp/etc/issue
~ftp/etc/msgs/welcome/etc/motd
/etc/ftpwelcome
/etc/issue if enabled via if= in /etc/gettytab
/etc/issue.net
/etc/ftpwelcome /etc/issue?/etc/default/telnetd
/etc/default/ftpdn/a/etc/issue?/etc/issue
/etc/motd telnetd, ftpd bannerset date/time
(from net: ntp or other)ntpdatedate?date
adjkerntz
ntp
tzsetup
(obs) timed
bsdconfig (10+)
ntpdatentpdatetimeslaventpdate
rdate
netdate System Preferences/
Date & Time ?date
ntpdate
ntpd
timed (obs)date
rdate
ntpd
timed??ntpdate
rdate ?ntpdaterdate?set date/time
(from net: ntp or other)TASK \ OS AIXA/UXDG/UXFreeBSDHP-UXIRIXLinux
Mac OS XNCR UnixNetBSDOpenBSDReliantSCO OpenServer
SolarisSunOS 4Tru64
UltrixUNICOSOS / TASK auditingaudit
auditpr
auditselect
lsaudrec??auditdaudisp
audevent
audomon
audsyssatd
satconfig
sat_reduce
sat_interpret

(2.6+) auditd
/var/log/faillog

???

mtree

/etc/security
(run nightly)

??audit
auditd
auditreduce
praudit    auditingencrypted passwords in /etc/security/passwd/etc/shadow/etc/shadow/etc/master.passwd/etc/passwd (not Trusted)
/.secure/etc/passwd (Trusted 9)
/tcb/files/auth/
(10+)
/etc/shadow (11.22+)/etc/shadow/etc/shadow (may vary)NetInfo/etc/shadow/etc/master.passwd (/etc/spwd.db)/etc/master.passwd (/etc/pwd.db)/etc/shadow/etc/shadow/etc/shadow/etc/passwd/tcb/files/auth (only with Enhanced Security)/etc/shadow/etc/udbencrypted passwords in min password length /etc/security/user??/etc/login.conf/etc/default/security?/etc/pam.d/system-auth???/etc/login.conf??/etc/default/passwd????min password length allow/deny root
logins/etc/security/user?/etc/default/login/etc/login.access
/etc/ttys /etc/securetty /etc/default/login/etc/securetty
/etc/ttys (doesn't affect ssh)/etc/default/login/etc/ttys/etc/ttys/etc/default/login/dev/default/login/etc/default/login/dev/ttytab/etc/securettys/etc/securettysudbrestrict -Rallow/deny root
loginsfirewall configgenfilt
smitty ipsec4
wsm network ??ipfw
ipf
pfctl/etc/opt/ipf/ipf.conf/etc/ipf.confiptables
ipchains
ipfwadm
(rh) redhat-config-
securitylevel Aqua / System Prefs /
 Sharing / Firewall
ipfw ?

ipf
/etc/ipf.conf
/etc/ipf6.conf

pfctl (3.0+)
/etc/pf.conf (3.0+)

pfctl
/etc/pf.conf ??

(10+) ipf, ipfs, ipfstat
/etc/ipf/ipf.conf

(9+) /usr/aset/ (hardening tool)
(9) SunScreen 3.2

????firewall configTASK \ OS AIXA/UXDG/UXFreeBSDHP-UXIRIXLinux
Mac OS XNCR UnixNetBSDOpenBSDReliantSCO OpenServer
SolarisSunOS 4Tru64
UltrixUNICOSOS / TASK show installed software lslpp -L
?pkginfopkg_info
ls /var/db/pkg
pkg info (10+)


rmfn, what (HP-UX 9)
swlist (HP-UX 10+)
swm list (11.31+)

versions
showfiles
showprods

(rh) rpm -a -i
(rh) rpm -qa
(rh) yum list installed
(deb) dselect
(deb) aptitude
(deb) dpkg -l
(gen) ls /var/db/pkg/*
(gen) eix -I

ls /Applications
Apple System Profiler
system_profiler
ls /Network/Applications
/Network/Users/*/
Applications
/Users/*/Applications

displaypkg
pkginfo  pkg_info pkg_infopkginfo -l
sysadm

swconfig (OpenServer 5)
displaypkg
pkginfo

pkginfo
prodreg
admintool
/var/sadm/install/contents? setld -isetld -i?show installed software file is in which package lslpp -w file
rpm -qif file
which_fileset ??

pkg_info -W file

pkg which file (10+)

swlist -l file | grep fileversions long |
grep file(deb) dpkg --search file
(deb) apt-file update && apt-file search file
(rh) rpm -qif file
(gen) equery b file
(gen) qfile file??pkg_chk
pkg_info -Fe filepkg_info -E file??pkgchk -l -p binaryToCheck????file is in which package add software

smitty install
rpm -ihv
installp -acgxYd /sourcedir[fileset|ALL]

n/a (no package system) sysadmsysinstall
/usr/ports make
pkg_add
pkg install (10+)
pkg add (10+)swm install (11.31+)
swinstallinst
swmgr (rh) rpm -hiv
(rh) yum install pkg
(deb) dselect
(deb) apt-get install pkg
(deb) dpkg -i
(md) urpmi
installpkg
drag-and-drop
/Applications/ Utilities/
Installerpkgadd
pkgplus pkg_add
/usr/pkgsrc && makepkg_add
/usr/ports && makepkgadd
sysadm

custom
installpkg
SCOadmin (GUI)

pkgadd?setld -lsetld -a /dev/rmt0h?add softwareprecompiled binaries of GPLware and freeware

perzl.org/aix/

bullfreeware.com

www.ibm.com/systems/power/
software/aix/linux/toolbox/altlic.html

see FAQ http://www.signal42.com/
xfree86/4.2.0/binaries/
DGUX-ix86/ftp.freebsd.org/
pub/FreeBSD
www.freebsd.org

hpux.cs.utah.edu

hpux.connect.org.uk

freeware.sgi.com
www.sgi.com/fun
nekochan.net

www.linux.org
linux.tucows.com
sourceforge.net
rpmfind.net
(deb) ftp.debian.org
(deb) packages.debian.org
(gen)packages.gentoo.org³
(gen)gentoo-portage.com³
(md)easyurpmi.zarb.org

fink.sourceforge.net

darwinports.opendarwin
.org

darwinports.com

?

ftp://ftp.netbsd.org/pub/
NetBSD/packages

ftp://ftp.NetBSD.org/
pub/pkgsrc

ftp://ftp.openbsd.org/
pub/OpenBSD/<version>/
packagesdata.mobach.nl/sinixwww.caldera.com/
skunkwarewww.sunfreeware.com
www.blastwave.org?www.tru64unix.compaq.com/
demos
tpegsrvr.zk3.dec.com/
freeware/tbl_packages.html??precompiled binaries of GPLware and freewareC compiler /usr/ibmcxx/bin/cc
/usr/vac/bin/cc($)cc
/usr/local/gnu/gcc/usr/sde/default/usr/bin/cc
gcc /usr/bin/cc (gcc, 9-)
/usr/bin/gcc (10+)
/usr/bin/clang(9.0+)/opt/softbench/bin/cc ($)
/bin/cc (9)
/usr/bin/cc (10+; not necessarily ANSI; kernel builder only)/opt/MIPSpro/
 bin/cc ($)
/usr/bin/cccc
gcc/usr/bin/cc (gcc)/usr/bin/cc
(Metaware) /usr/bin/cc (gcc) /usr/bin/cc/usr/bin/cc/usr/bin/cc/opt/SUNWspro/
 bin/cc²/usr/bin/cc/usr/bin/cc /usr/bin/cc/opt/ctl/bin/ccC compiler show patch level
and/or patches instfix -ivq
oslevel -s
emgr -l?what /dgux

uname -a

freebsd-version (10+)

swm list (11.31+)
swlist -l product |
grep PHsomeString

versions -b | grep patch
uname -R (6.5+)

(rh) rpm -q
(rh) cat /proc/version
(rh) cat /etc/redhat-release
(deb) dpkg -ssw_vers
Apple:About This Mac
ls /Library/Receipts
uname -a
lsbom
cat /System/Library/
CoreServices/
SystemVersion.plist pkginfo -lcat /kern/version
uname -a
/etc/release sysctl -n kern.versionuname -a
pkginfo | grep EKSswconfig -Pshowrev -p
prodreg (2.6+)
patchadd -p
cat /etc/releaseshowrev -p
(if you're lucky)dupatch -track -type kit
dupatch -track -type patch
setld -i | grep patchname
sizer -vB??show patch level
and/or patches patch tool installp
smitty update_all
emgr?dg_sysreport -p patches

csup (6.2+)
freebsd-update (7+)

cvsup or cvs to update,
then rebuild

svnlite (10+)

update (9)
swinstall (10+)inst
swmgr apt-get update
apt-get upgrade
(rh) up2date
(rh) yum update
(md) urpmiSystem Preferences / Software Update
softwareupdate pkgadd
pkgpluscvs
patch cvs
cvsup
sup
cvsync
patch
pkg_addpkgadd
sysadmpkgadd; pkgrm installpatch (2.5.1-)
patchadd (2.6+)
pkgadd; pkgrm
/usr/sadm/bin/smpatch (9+)
n/asetld -i
dupatch? ?patch tool configure/show
runtime linking
fortran-2000.com/
ArnaudRecipes/
sharedlib.htmldump -H
ldd ??ldd
ldconfig
kldstat chatr
ldd (11+)
kmadminldd
rld
_RLD_LISTldconfig
ldd
lsmod otool -Lldd
ldconfig ldd
nm
ldconfig ldd
ldconfig ldd?crle
ldd
pldd
modinfo
LD_PRELOADlddodump -Dl
ldd??configure/show
runtime linking
fortran-2000.com/
ArnaudRecipes/
sharedlib.htmllink library path$LIBPATH?$LIBPATH$LD_LIBRARY_PATH$LD_LIBRARY_PATH
(64-bit)
$SHLIB_PATH
(32- or 64-bit)$LD_LIBRARY_PATH
$RLD*_PATH $LD_LIBRARY_PATH
/etc/ld.so.conf

$DYLD_LIBRARY_PATH
man dyld

$LD_LIBRARY_PATH$LD_LIBRARY_PATH
/etc/ld.so.conf $LD_LIBRARY_PATH$LD_LIBRARY_PATH?$LD_LIBRARY_PATH
(7+ deprecated in favor of crle)$LD_LIBRARY_PATH$LD_LIBRARY_PATH?$LDPATHlink library pathtracing utility probevue (6+)
trace
syscalls
truss
watch
trcrpt??=dg_strace
(>=4.20MU06)ktrace
truss
dtrace (8+)
dtruss (8+) caliper fprof
caliper ktrace
trace (freeware)
tusc (11+, freeware)par
prfstat
SystemTap(freeware)

strace
ltrace

ktrace; kdump (<=10.4)
dtruss; dtrace (>=10.5)
fs_usage trussktrace
ktruss
kdump ktrace
kdump
systracetruss
scstattruss

truss
sotruss
dtrace (10+)

tracetrace
trusstracecdbx
jumptrace tracing utility define user defaults/etc/security/
mkuser.default
/etc/environment /etc/profile
/usr/lib/skeladmuser -o set/etc/login.conf/etc/default
/etc/PATH
/etc/MANPATH
/etc/default/*

/etc/default/login
/etc/profile

/etc/profile
/etc/security/
/etc/skel/
/etc/profile.d/*/usr/bin/defaults/etc/default/login
/etc/profile
/etc/security/ /etc/profile
/etc/skel/*
login.conf

/etc/profile
/etc/skel/*
/etc/login.conf

/etc/default/login
/etc/profile
$HOME/.profile ?/etc/default/login
/etc/profile
/etc/security/ /etc/profile/etc/skel
/etc/profile /etc/skel
/etc/profileudbgen
nu scriptsdefine user defaultscsh global .login/etc/csh.login??/etc/csh.login/etc/csh.login/etc/cshrc/etc/csh.login/etc/csh.login/etc/.login/etc/csh.login/etc/csh.login??/etc/.login?/etc/csh.login?/etc/logincsh global .logindefault syslog and messages/var/adm/syslog
/var/adm/messages ?/var/adm/messages/var/log/messages/var/adm/syslog/syslog.log
/var/adm/sulog
/var/spool/mqueue/syslog/var/adm/SYSLOG
/var/adm/sulog /var/log/syslog
/var/log/messages
/usr/adm/messages
/var/log/maillog /var/log/system.log/var/adm/streams
/var/adm/usererr
/etc/.osm /var/log/messages/var/log/messages/var/adm/log/messages/usr/adm/syslog
/var/adm/log/osmlog

/var/adm/messages
/var/log/syslog
softpanorama.org/Logs/solaris_logs.shtml

/var/adm/messages

/var/log/syslog (sendmail)

(4.x+) /var/adm/messages
/var/adm/syslog.dated (kern, auth, daemon, lpr, syslog, user)
/var/adm/binary.errlog /var/adm/messages
/var/adm/syslog.dated (kern, auth, daemon, lpr, syslog, user)
/var/adm/binary.errlog/usr/adm/messagesdefault syslog and messagessystem error reporting tool errpt
errclearerrptadmsyslog -o reportdmesg
send-pr evmshow (11.31+)
smh (11.31+)
dmesg
sysdiag (9 and early10)
stm/cstm/mstm/xstm (10.20+)
EMS (from support CD)sysmon
amsyslog
availmon
eventmond
imdmonitor
syserrpanel

dmesg

(deb) reportbugConsole
/var/log/crash.log, dmesgodm
errpt dmesg
cat /kern/msgbuf
send-prdmesg
sendbugsysadm
log3
WebSysAdmindmesgprtdiagdmesg(4.x) uerf; dia (DECevent)
(5.x) (System errors) EVM (evmget; evmshow)
(Hardware errors) dia; ca (Compqaq analyze)uerferrptsystem error reporting tool performance monitoring

ftp://ftp.software.ibm.com/
aix/tools/perftools/perfpmr

vmstat; iostat; sar; filemon; topas; trace; svmon; netpmon;
proctree

?sar
netstat
uustat fstat; nfsstat; systat; vmstat; netstat; uustat (9-); top; iostat;
gstatglance
gpm
caliper
top; sar; vmstat;
iostat; nfsstat; netstat;
uustat; Glance($)
ptree (11.31+) pcp; dkstat; pmkstat;
sar; osview; gr_osview;
ecstats; linkstat; netstat;
nfsstat; nfsvis; nodevis;
routervis; nstats; topvmstat
procinfo -D
top
htop
pstreevm_stat; nfsstat;
fs_usage; sc_usage;
top
Activity Monitor (10.3+)
Process Viewer (10.2-)sar; pinfo -v;
modstat; netstat;
nfsstat; pppstat;
smtpstat; snmpstat;
uustat; top
iostat; netstat; systat;
vmstat; top; fstat;
nfsstatfstat; iostat;
netstat; nfsstat; pstat;
systat; top;
vmstat; vmstat; mpstat; dkstat;
sar; netstat; scstatsar

sar; iostat; kstat;
mpstat; netstat; nfsstat;
prstat; trapstat; vmstat; ptree

iostat; netstat; vmstatiostat; nfsstat; vmstat
/usr/opt/sv4b/bin/sar
monitor iostat; nfsstat; vmstat
/usr/opt/sv4b/bin/sar

sar; xsar; tsar; xsam;
xscc; xns; xnetmon;
nfsstat; perfmon; ja;
mppview; xmppview;
sam; csam

performance monitoringmatch process to file or portprocfiles (5.2+, files only)?fuser sockstat -llsof
fuser fuserlsof
netstat -atuplsoffuserfstat
sockstatfstatfuser?fuser
lsof^2
pfiles ????match process to file or portmanage various logssnap?????logrotate
(rh)sysreport; sosreport??????logadm
/opt/SUNWexplo/bin/explorer????manage various logsX pop-up/usr/dt/bin/dterror.ds??/usr/local/bin/xmessage/usr/dt/bin/dterror.ds/usr/bin/X11/xconfirm/usr/X11R6/bin/xmessageosascript -e 'tell application "Finder" to display dialog "msg"'?xmessage/usr/X11R6/bin/xmessage??/usr/dt/bin/dterror.ds????X pop-upWikipedia AIXA/UXDG/UXFreeBSDHP-UXIRIXLinux
Mac OS XNCR UnixNetBSDOpenBSDReliantSCO UnixWare
SolarisSunOS 4Tru64
UltrixUNICOSWikipediaFAQs (see also faqs.org)www.faqs.org/faqs/
aix-faq/part1/
preamble.htmlchristtrekker.users.sourceforge.net/
doc/aux/faq.htmlwww-csc.dg.com/csc/
dguxfaqs1.aspwww.freebsd.org/doc/
en_US.ISO8859-1/books/faqwww.faqs.org/faqs/
hp/hpux-faq/preamble.htmltechpubs.sgi.com/library/
tpl/cgi-bin/browse.cgi?coll=0650
&db=FAQ

www.linuxdoc.org/
FAQ/Linux-FAQ

www.debian.org/doc/
FAQ

www.darwinfo.org/
faq.shtml

www.macosxhints.com

?www.netbsd.org/
docswww.openbsd.org/faq?pcunix.com/SCOFAQ/http://opensolaris.org/os/about/faq/
www.faqs.org/faqs/Solaris2/
FAQ/
www.faqs.org/faqs/Solaris2/
x86/FAQ/
www.stokely.com/
unix.sysadm.resources/
faqs.sun.html
sunhelp.orgwww.faqs.org/faqs/
comp-sys-sun-faq/www2.tru64.org/faq/
tru64_faq.phpwww.supelec.fr/decus/
faq/faq-ultrix.htmlwww.spikynorman
.net FAQs (see also faqs.org)blogs???blogs.freebsdish.org? linuxmoz.com???????????blogsmailing list AIX-L@pucc.
princeton.edu
(LISTSERV)n/a? lists.freebsd.org/mailman/listinfo
hpux-admin@
dutchworks.nl
 (majordomo)
hp-ux-l?

www.linumanagers
.org

www.linux.org/
help/lists.html

lists.apple.com?www.netbsd.org/
MailingListswww.openbsd.org/
mail.html??sun-managers@sunmanagers
.org (majordomo)sun-managers@
sunmanagers
.ececs.uc.edu
(majordomo)tru64-unix-man
agers@ornl.gov   (majordomo)
http://www.ornl.gov/its/archives/
mailing-lists/decstation-man
agers@ornl.gov
 (majordomo)?mailing list mailing list archives 

www.marist.
edu/htbin/wlvindex?
ADSM-L

marc.theaimsgroup.com/
?l=aix-l&r=1&w=2

n/a?lists.freebsd.org/mailman/listinfowww.dutch
works.nl/
 htbin/hpsysadmin ? www.linuxmanagers.org/
pipermail/linuxmanagers
lists.debian.orglists.apple.com?www.netbsd.org/
MailingListswww.openbsd.org/
mail.html??

www.sunmanagers.org/
archives.html

www.latech.
edu/sunman.html

www.latech.
edu/sunman.html www-archive.
ornl.gov:8000www-archive.
ornl. gov:8000?mailing list archives 

man pages

www.freebsd.org/
cgi/man.cgi

publib.boulder.ibm.com/infocenter/
aix/v7r1/topic/ com.ibm.aix.cmds/
alphabeticallistofcommands.htm

n/a?man.freebsd.org
docs.hp.com/en/hpuxman_pages.html
hpuxman_pages.htmltechpubs.sgi.com/
library/tpl/cgi-bin/browse.cgi?
coll=&db=man

www.linuxmanpages.com/

Fedora Core 4:
linuxcommand.org/
superman_pages.php

www.tldp.org/docs.html#man

developer.apple.com/
documentation/Darwin/
Reference/ManPages?man.netbsd.orgwww.openbsd.org/cgi-bin/man.cgi?uw714doc.sco.com/en/Navpages/docs.sun.com/
app/docs/
coll/40.10????

man pages

www.freebsd.org/
cgi/man.cgi

newsgroup(s) and forums
groups.googlecomp.unix.aix
www.thp.uni-duisburg.de/
cuaix/cuaix.htmlcomp.unix.auxcomp.os.aoscomp.unix.bsd
freebsd.misc
forums.freebsd.orgcomp.sys.hp.hpux comp.sys.sgi.admin comp.os.linux.* (esp .answers)comp.sys.mac.*comp.sys.ncrcomp.unix.bsd.
netbsd.misc comp.unix.bsd.
openbsd.misc
lucky.openbsd.miscde.comp.os.unix.sinixcomp.unix.sco.misccomp.sys.sun.
admin, comp.unix.solaris comp.sys.sun
admin comp.sys.dec,comp.unix.tru64comp.unix.ultrixcomp.unix.cray

newsgroup(s) and forums
groups.google

user groupswww.userblue.orgn/a?www.freebsd.org/
support.html#userconnect-community.org
hp-interex.com
encompassus.org
hptechnologyforum.com
forums.itrc.hp.com
HP DSPP?www.debianhelp.orgwww.apple.com/
usergroups
www.omnigroup.com/
developer/mailinglists
www.themacintoshguy.
com/lists/X.html ?www.netbsd.org/gallery/
groups.htmlundeadly.org??opensolaris.org/os/community/
advocacy/usergroups/ug-leaders/?Encompass/DECUS?www.cug.org
www.excray.comuser groupsmagazines

n/a

n/a?
daemonnews.org
bsdmag.org
freebsdnews.net
bsdnow.tv (videos)
bsdtalk.blogspot.com (audio podcasts) www.hpchronicle.com
www.interex.org/hpuxusr
www.hppro.comwww.sgi.com/support/
pipeline.htmlwww.linuxjournal.com
www.linux-mag.com
www.debianplanet.orgwww.macworld.com
www.stepwise.com?www.daemonnews.comundeadly.org??www.sunworld.com
sun.com/sun-on-net/itworld?www.tru64.org??magazinesvendor home pagewww.developer.ibm.com/
tech/map/aixrm.html
www-941.ibm.com/
collaboration/
wiki/display/WikiPtype/Home
n/a

www-csc.dg.com/
csc/default.asp

www.freebsd.orghp.com/go/hpuxwww.sgi.com/software/
software.html#IRIX www.linux.org
www.debian.org
gentoo.org
www.redhat.com
(SUSE) www.novell.com/linuxwww.apple.comncr.com
teradata.comwww.netbsd.orgwww.openbsd.orgwww.fujistu-siemens.comwww.caldera.com/
products/openunixsun.com/solaris
sun.com/bigadmin
sun.com/developers/support
soldc.sun.com
access1.sun.com
opensolaris.org?www.tru64unix
.compaq.comwww.supelec.fr/decus/
faq/faq-ultrix.htmlwww.cray.comvendor home pagevendor docs and patches (see also man pages)

www.redbooks.ibm.com
www.ibm.com/
developerworks/aix
techsupport.services.
ibm.com/
server/support?
view=pSeries

?www-csc.dg.com/csc/
custdocsasp/
custdocs2.asp?
category=069docs.asp

www.freebsd.org/handbook
www.freebsd.org/releases/
www.freebsd.org/security
www.freebsd.org/doc/en/books
www.freebsd.org/doc/en/articles

software.hp.com
docs.hp.com
itrc.hp.com

techpubs.sgi.com
www.sgi.com/
support/patch_intro.html
support.sgi.com
relnotes
grelnotes

tldp.org

www.linuxdoc.org/
rpmfind.net/linux/RPM

developer.apple.com/
darwin
www.info.apple.com/
support/downloads.html
www.apple.com/support/
security/security.html

www.info.ncr.com/
eCRM-
OSHW-Browse
By-Results.cfm
?pl=&PID=&title=
&release=
&pl=Software
%20-%20
MP-RAS&sbrn=3
www.netbsd.org/
Security

www.openbsd.org/
docum.html

www.openbsd.org/
errata.html

www.openbsd.org/
security.html

service.fujitsu-siemens
.com
its.siemens.com/lobs/its/
its_sc/eks_en/unix_en.htm www.caldera.com/support

docs.sun.com
sunsolve.sun.com

?www.tru64unix
.compaq.com/docsN/A (EOL)www.cray.com/
craydocvendor docs and patches (see also man pages)vendor phone (US) 800-426-7378
(800-IBM-SERV) (EOL; unsupported) ?925-674-0783
(FreeBSD Mall) 800-633-3600800-800-4SGI?1-800-MY-APPLE???(Germany)
+49-5251-80?800-USA-4SUN?800-344-4825800-344-4825?vendor phone (US)wikiswww.ibm.com/developerworks/
wikis/display/WikiPtype/Home??wiki.freebsd.org??fedoraproject.org/
wiki/Main_Page(fed)
susewiki.org(SUSE)wiki.osx86project.org??www.openbsd-wiki.org??wikis.sun.com
www.big-up.org/menu/wiki/doku.php????wikisTASK / OSAIXA/UXDG/UXFreeBSDHP-UXIRIXLinuxMac OS XNCR UnixNetBSDOpenBSDReliantSCO OpenServerSolarisSunOS 4Tru64
(Digital Unix, OSF/1)
(4) 4.0F/G
(5) 5.0AUltrixUNICOSOS \ TASK 

Depending on how you’ve chosen to look at technology developments, you may find reaching out to customers way easier than before. And you’d be right too, as helpful tools alike email and social media are used in conjunction to spread messages across all geographies. Nonetheless, with email marketing no longer being done by the book, marketers are becoming more challenged to wrap up a working strategy.

What fuels the success of mobile marketing?

Social media platforms have a dozen distinctive advantages in the marketing world, and enjoy an overwhelming gratitude for having substituted dysfunctional traditional campaigns. Still, email marketing remains the method that produces the highest return of investment (122%) in this sector, and the one that generates the most confidence and reliability among customers. If you still don’t have an effective email marketing software by hand, 2017 is the right time to look for it.

According to a McKinsey.com study on the benefits of email marketing, promotional emails are preferred by more than 90% off US adults. The same study concluded that email marketing strategies are approximately 40% more effective than promoting on Facebook and Twitter, and that despite of social platforms being the masters of non-formal approach, emailing clients does make marketing as personal as you need it.

What are the reasons to use email marketing software?

These tools build and enhance credibility, and make people like you, trust you, and check your services even when they don’t need them. They expand your reach on multiple geographies and demographics at a time, and help sell to audiences who may as well never have heard of your product, even on a mobile device. They are your safe two-way street for learning what customers need, and for strengthening relationships with them, as well as a chance to present yourself in a professional light regardless of how well you’re doing. That’d be a bit tougher to achieve on a social platform.

Besides, today’s best email marketing tools are the only products that allow bulk distribution of personalized content, and are thus a powerful arm to reengage existing customers. To help you find one that works for you, we’ve listed the 20 best email marketing tools for small, medium, and large businesses, with a detailed overview of how they can be helpful to you.

These are the 20 best email marketing tools to consider in 2017:

1. HubSpot Marketing
2. RedCappi
3. MailChimp
4. AWeber
5. GetResponse
6. SendGrid
7. Mailjet
8. Campaign Monitor
9. ActiveCampaign
10. Elastic Email
11. Marketo
12. Autopilot
13. Zoho Campaigns
14. Benchmark
15. Robly
16. iContact Pro
17. Adestra
18. Klaviyo
19. FreshMail
20. Sendinblue

1. HubSpot Marketing

What HubSpot has to offer it’s an all-in-one, first-in-class marketing suite for inbound marketers and teams, where email marketing features tell only a part of the story. Behind the scenes, HotSpot will automate most of your vital marketing operations, put in place adept social media management, and make sure you generate as much demand for your products as you need. This makes it the leader of our Marketing Software category, with a score of 9.0/10, and a user satisfaction rate of incredible 99%.

To start with, HubSpot will help you prepare and optimize promotional content, and place it in front of the right audiences. Meanwhile, you will find it much easier to engage those audiences given that your messages are highly personalized, and adjusted to the persona and expectations of the future consumer. It is at this point that HubSpot Marketing takes over the entire purchase journey, allowing you to organize drip campaigns and queue up a series of emails that keep your leads engaged until they buy. Due to its integration with HubSpot CRM, the platform will also record valuable customer information, and pull out stats you can use to report on your impact. To make the entire process even more interesting for your team, HubSpot Marketing will allow you to design your own web pages and attract targeted traffic towards them, optimize content for search engines, and turn your sales management into a hands-off operation.

What is unique about HubSpot Marketing?

1. End-to-end control over your campaigns and workflows. HubSpot Marketing is a compact solution that will let you design your own workflows, prepare, format, and publish content, and seamlessly integrate contact profiles and information to make the most of your campaigns.
2. Powerful lead evaluation. In its role of a robust marketing funnel, HubSpot Marketing collects and imports valuable contacts from a variety of sources, and lets you move them to the appropriate stage. At the same time, it evaluates them based on demographics and behavior, scores and nurtures them, and packs all important traffic data in understandable visuals.
3. Versatile email marketing. With HubSpot Marketing, you can create both static and dynamic HTML messages, customize them in accordance with your customers’ persona and behavior factors you believe could give you the response you want.
4. Social intelligence. Rather than focusing solely on email marketing, HubSpot Marketing governs and tracks activities on social media, and provides valuable intelligence on your most qualified leads. Instead of preparing content and posting it separately on each platform, you can now manage all of them from a single dashboard.
5. Easy landing page builder. HubSpot Marketing has one of the easiest and most intuitive landing page builders, and requires no coding skills at all. You can use any of the system’s beautiful and ready-to-use templates, or draft one of your own.
6. SEO optimized content. HubSpot Marketing does not only help post and distribute content, but also prepares it to make it more accessible to interested audiences. For the purpose, it will let you work with a keyword filter and other modern SEO tools, and ensure you’re one step ahead of your competitors.
7. Integration with other HubSpot products. The ideal target team for HubSpor Marketing is one that already employs the platform’s CRM or Sales services, as the three products are completely synchronized, and let data flow seamlessly from one to the other.

2. RedCappi

RedCappi is another of our leading email marketing apps, and the role model of successful new comers in the productivity software industry. With its functionality packed in a cute interface you’d certainly remember,  RedCappi attracted the attention of numerous small and medium agencies, but also sole marketers looking to manage a larger database of customers and contacts. What they all liked about the app is its unparalleled simplicity. There is also a great free trial available if you want to try out the key features of the software first. You can easily sign up for RedCappi free trial here.

What makes RedCappi truly valuable is not only how it manages the contacts you have, but rather its role in enlarging that list. The tool lets you create catchy web signup forms with only few clicks, and imposes no restrictions to as where you can embed those. You can drag-and-drop images, icons, coupons, and even videos to make your emails more attractive, and track with ease who opened them or who subscribed/unsubscribed from using your services. In such way, RedCappi exposes the reasons for risky bounce rates, and does so much cheaper than acquiring a business intelligence service to complete the task. As expected, the platform will deliver just enough stats and analytic capacity to measure the progress of each campaign.

What is unique about RedCappi?

1. Launching creative campaigns. RedCappi offers one of the easiest campaign creators on the market, where you can either prepare your own content, or modify any of its beautiful templates. For the purpose, you will only need to drag-and-drop the desired elements on the template, including text, images, social media icons, and coupons.
2. One-to-one contact management. RedCappi won’t only collect your contacts, but also keeping them interested and engaged. With a variety of modern tools, RedCappi helps marketers personalize communication with their leads, and segment those lists in a way they deem appropriate.
3. Lead capturing. Using RedCappi, you can create forms and embed them on your website to get new leads, including both the mandatory and the custom fields you need to familiarize with your customers. The tool will also allow you to confirm submissions with emails and thank-you pages, or by simply adding subscribers to your conversion lists.
4. Management of social interactions. RedCappi plays an important role in enlarging your customer list, and generates new ones from the social networks where you’ve posted your email  campaigns. In this way, you can raise brand awareness, boost customer loyalty, and share content with wide audiences before they’ve even visited your store.
5. Real-time insights. RedCappi provides insights that help understand which practices work the best, and which customers are more likely to buy. You can use the same system to compare the progress of different campaigns, and make smarter decisions.

3. MailChimp

MailChimp is probably the world’s best known email marketing service, and for a good reason. Rather than helping you keep the pile of contacts and partners organized, MailChimp makes sure that you will shed some professional light on your brand, and sell more regardless of how far your marketing career has progressed. It ease of use is easily comparable to the one of RedCappi, and the functions of the two systems are quite similar as well. The plus-factor on MailChimp’s side would be its robust experience in the area, and the number of satisfied users who believe they’d perform much weaker without it.

What is it that makes MailChimp special? Explained in the simplest way, MailChimp let’s you prepare email campaigns that look just the way you’ve imagined. There is little you can’t tweak to convey the exact message you have in mind, which consequently dictates how viewers interact with your news, services, and products. The same as with RedCappi, you can drag-and-drop items to design suitable signup forms, and embed those on different locations.

What is unique about MailChimp?

1. A faithful campaign building assistant. MailChimp is the tool that collates all necessary tools and sources for you  to build campaigns that genuinely convey your messages. Next to the fully featured content editor and endless choice of suitable templates, MailChimp will also provide you with learning materials, tests, and experts’ assistance.
2. End-to-end automation for online sellers. MailChimp’s main role is to ensure that you will reach the right audiences at the right time, and in line with their needs, preferences, and expectations. With this system, you will only need a single API request to segment contacts and trigger a custom series of emails, and report separately on each of them.
3. Advanced analytics. MailChimp keeps an eye on your website and sales activity, and drafts all types of standard reports that can help you refine your future campaigns.
4. Ad campaigns on social media. Using MailChimp, you will be able to organize ad campaigns on social media (Facebook and Instagram), and simplify sales processes for your customers.
5. Mobile apps. MailChimp comes with specially adjusted mobile apps for different devices, where users can prepare and send campaigns, check on their reports, or add notes as they usually do when in the office. This way, MailChimp helps enhance the productivity of decentralized and remote.

4. AWeber

AWeber is one of the top performing enterprise-friendly email management solutions trusted and relied on by over 100,000 users worldwide. It provides an array of interaction management functions packed in a well-structured marketing story that simply takes you from introduction to purchase with minimal effort. It starts by preparing highly personalized and engaging emails on plain-text and HTML editors, and setting up automated series so that you will keep customers interested regardless of their current status. Its industry-leading deliverability is a guarantee that you can reach the audiences that matter, be those local buyers or international customers. Another thing to appreciate about AWeber are its streamlined integrations with leading productivity systems, including names such as PayPal, WordPress, Facebook, and many more.

What is unique about AWeber?

1. Scheduled and automated emailing. With AWeber, users can automate all of their campaigns and send emails to tagged and targeted subscribers. Emails can be scheduled and bulk-sent automatically, and customers can be reengaged with drip campaigns.
2. Subscriber segmentation. AWeber targets your subscribers depending on their behavior and actions, and uses this info to segment them in suitable categories.
3. Autorepsonder follow up. AWeber helps you turn casual subscribers into paying customers, by setting a simple sequence of emails and keeping the audience engaged with welcome series, product launch series, and customer onboarding sequences.
4. First-in-class customer support. The company deploys a team of email marketing professionals who are highly knowledgeable, responsive, and effective. For those who are new to email marketing, AWeber offers regular live webinars for added instruction and training.
5. All of your favorite integrations. AWeber connects seamlessly with more than 100 third-party applications and services, which means you can use it in conjunction with the rest of your ecosystem. The list of integrations includes OptinMonster, Interact, Unbounce, and many others.

5. GetResponse

For many experts in the niche, GetResponse defines in both form and function how a truly successful email marketing software should look like. From our perspective, it may as well be qualified as the most visual tool of its breed, and one that offers premium quality on a price even SMBs can afford. GetResponse is most recommended for the neat combination of different marketing functionalities, as the same platform can be put in action to create and deliver compelling emails, organize marketing webinars, publish highly-converting landing pages, and automate your customers’ journey depending on what they need and how they behave. This makes AWeber a very compact and intelligent solution for all industries, especially those depending on direct communication with end customers. Hosted in cloud, AWeber won’t be a burden to set up and maintain, and results will be visible almost immediately.

What is unique about GetResponse?

1. Powerful content design. Regardless of how much your email strategy has improved, GetResponse will help reinvent it in only few steps. For starters, you have unlimited design possibilities to produce engaging content, 500 ready-to-use templates, and 1,000 free stock images. Dynamic ads that include contact info and custom details can be controlled with the mobile preview function, and distributed among users in the right time.
2. Autopilot marketing communication. What users appreciate the most about GetResponse is the possibility to schedule emails, and trigger autoresponders for their client messages. They can use a handy drag-and-drop organizer to manage also their e-courses, customer tips, and other educational cycles.
3. Advanced segmentation features. Regardless of how many contacts you need to manage, GetResponse builds lists and promotes engagement with ease. You can group similar contacts in lists and categories, and send relevant and highly personalized messages.
4. Perfect timing. Perfect timing is a dedicated delivery optimization feature you can use to send messages when customers are most active, and prepare only content you know they’d be interested in. The Time Travel feature, on the other hand, tracks delivery, and makes sure that messages are promptly distributed with time zones considered.
5. Real-time activity tracking. GetResponse is also an excellent alternative for analytics, given that it provides you with a bird-eye-watch on all contacts and interactions, and compares the success of your campaigns completely independently.

6. SendGrid

SendGrid is another system that specializes in streamlined email deliverability. With an absolutely free signup, 40k  free emails per month, a 5-minute process of setup and integration, and reliable support, SendGrid represents a moneyworth investment in the definite sense of that phrase.

At first sight, SendGrid reminds of a fully featured digital asset designer, as it offers a large suite of content modification tools that give it a competitive edge in this industry. Basically, SendGrid is the way to go for all creative marketers who’d like to offer users modern and unique content. Yet, this is not where SendGrid ceases its mission – The suite consists of quality IP streams governed by 30+ deliverability experts, which makes it possible to reach and engage the right customers at any point of time. It also offers one of the best analytic kits you can use to create email success reports, IPS, device, and geography summaries.

What is unique about SendGrid?

1. Transactional email management. SendGrid provides cloud-based email delivery services that assist companies with transactional email management while at the same time abiding by anti-spam regulations. The tool manages various types of email such as friend requests, shipping notifications, email newsletters, and sign-up confirmations.
2. Master of scalability and deliverability. Known as the most scalable and restrictions-free email management system, SendGrid has proven being able to deliver a huge volume of email effectively. Currently, the app sends 4,190 emails per second.
3. Security and monitoring. Committed to anti-spam laws and regulations, SendGrid will support the professionalism and good reputation of your service with tools such as Internet service provider (ISP) monitoring, sender policy framework (SPF), domain keys, and feedback loops.
4. Integrations with leading mailbox providers. SendGrid’s development team has fostered close relationships with all leading mailbox providers – Gmail, Yahoo and Outlook.com. Together they work with SendGrid via an industry association called MAAWG – the Messaging Anti-Abuse Working Group to fight spam, phishing, denial of service attacks, mobile spam and other forms of email abuse.
5. Dedicated customer support. SendGrid offers 24/7 customer support and a dedicated team of customer success managers who deliver personalized support and guidance to its customers. Available via phone, chat, and email, SendGrid’s team of 25 technically trained.

7. Mailjet

Mailjet is Europe’s favorite email marketing tool that offers solutions for both marketers and developers, as well as enterprise-grade teams. Its highlight is, as the company likes to put it, business class & transactional email sending, namely the preparation and distribution of personalized and optimized emails that help sell more. Wiith Mailjet, you will receive a dedicated IP address to stay in full control of your sender reputation, an intuitive and comprehensive editor (Passport) for your emails, A/B tests on calls-to-action and comparison of the performance of your campaign, and SMTP Relay to send transaction emails and track customers with ease. During the process, you will have the peace of mind that Mailjet has handled all contact management operations, and that there is always a marketing expert on the other side of the line ready to assist you.

What is unique about Mailjet?

1. Customer-oriented dynamic messaging. Mailjet allows you to customize dynamic content, blending the variables different for every user such as location, surfing habits or name etc. That’s how you attract every customer’s attention towards your marketing campaign.
2. No-spam guarantees. Mailjet enhances deliverability, and ensures that no message you send will end up lost or stuck inside the Spam folder. The reason for that is that your correspondence and customer relationships are managed by a team of experts and marketing professionals.
3. MJLM and Passport Responsive email builder. Mailjet helps you create beautiful and fully optimized content, and hand it out to particular groups of users. The systems’s Send APIs are quick and reliable, and make it possible to automate welcome messages, account notifications, and shipping confirmations.
4. Newsletter templates. Mailjet comes with a unique library of pre-made newsletter templates which you can edit and personalize using the Passport content builder.
5. SMTP Servers & REST API. You can use Mailjet’s SMTP Service and REST API to import contacts, and communicate with clients on as many platforms as you need. The company offers comprehensive documentation to help you connect the system independently.

8. Campaign Monitor

Another brand in the industry that you should definitely consider is Campaign Monitor, a powerful email marketing and automation system designed for personalized campaigns. The system is robust, yet instantly understandable, and easy to tailor so that you can achieve the exact business results you need. Inside, you will find a number of beautiful and fully-functional templates you can easily edit and repurpose, and track the success of your campaigns regardless of the device you’re using. Campaign Monitor lets you organize email data and use its list management tools to create segments and target the audiences you need, as well as to test and optimize those segments until you’ve created the pitch perfect material for your customers. To make matters even better, Campaign Monitor will report in real time how customers are interacting with your email, and be of great help to decide which part of your campaign is not progressing well.

What is unique about Campaign Monitor?

1. An effective DIY email builder & optimizer. Campaign Monitor’s drag-and-drop email builder offers a range of professionally-designed templates that you can use to create your own branded and unique emails that will look wonderful on any device. The app’s DIY email builder optimizes personalized emails for every inbox, and the reader can use a desktop or mobile device to read the messages.
2. Accurate segmentation. Campaign Monitor’s advanced segmentation features enable you to send relevant messages to the most promising prospects who look like they are likely to convert. These features are: subscriber segmentation, message triggers, content personalization, webhooks, and many more.
3. Effortless campaign automation. You can set campaigns up and import your subscriber and customer information into Campaign Monitor. Then, sit back and relax as this cloud-based app sends pre-designed messages at the right times.
4. Campaign analytics. Campaign Monitor offers interactive analytics that provides insights on how your emails are performing in real time. You can see how your readers are engaging with and sharing your content, and how effective your call to actions are. Campaign Monitor also offers robust analytics and reporting components that allow you to see who opened your emails and what aspects of your content are most interesting to readers. You can even see with whom the emails are shared with.
5. Social sharing. You can use Campaign Monitor to share and promote your posts on social media platforms, among which Facebook and Twitter.

9. ActiveCampaign

If looking to automate vital marketing operations and optimize the success of your email activities, look no further than ActiveCampaign. The platform will not only help you deliver the right message in the right time, but also be of great use to research on your audience, and create that message in the first place. For the purpose, ActiveCampaign delivers detailed market insights, pull out valuable customer data, and makes sure that all your contacts and partners are managed from the same location. It will be your marketing, sales, and CRM bundle that consolidates communication and moves it ahead, as you will conclude from the instant improvement of your team’s collaboration. Next to email marketing, this platform offers also site messaging, SMS, event tracking, and a fully integrated form builder.

What is unique about Active Campaign?

1. Built in CRM and sales automation features. Active Campaign ties its email and messaging features to a user-friendly and robust marketing automation platform, along with built-in CRM and sales automation functionalities. In such way, it enables businesses to create special marketing workflows as well as automated follow-up sequences, tracking at the same time the behavior multiple marketing channels.
2. Conditional content. With Conditional Content you are able to dynamically alter content on a per contact basis. For instance, you could display entirely different images or blocks of text in an email based on how a contact is tagged or what web pages they have viewed on your website.
3. Site tracking. Site Tracking connects your marketing to a contact’s behavior allowing you to respond to what they are doing on your website and adapt your marketing to their interests. This feature allows ActiveCampaign to “see” what a contact is doing on your website. Their entire page view history is attached to their contact record so you know what they have and haven’t done on your website.
4. Event tracking. You could use Event Tracking to notify ActiveCampaign when a user logs into your website, or when they make a transaction, and then trigger a follow up campaign.
5. Unlimited integrations. With an open API infrastructure, ActiveCampaign will blend seamlessly in any software environment. Plus, you will get access to 150+ native integrations, among which leading accounting, CMS applications, and analytics.

10. Elastic Email

We recommend Elastic Email as a highly configurable solution that meets the needs of businesses of all scales, and packs all essential deliverability tools modern marketers to monetize their communication with customers. With Elastic Email, it takes literally few clicks to prepare attractive emails (HTML editor), or even less had you decided to transform any of the platform’s ready-to-use templates. Meanwhile, Elastic Email will keep an eye on your campaigns and measure their impact, so that you can make more informed decisions in future. Their reports are easy to use, and contain platy of visual charts and graphs that make it possible to compare campaigns, while the suite of highly adaptive API helps blend Elastic Email inside virtually any software ecosystem.

What is unique about Elastic Email?

1. Drag-and-drop & HTML editor. You can either choose a suitable pre-built template from Elastic Email’s library, or code one that will convey your message.
2. Autoresponders and drip campaigns. Elastic Email focuses on helping you reaching clients even when you’re not behind the screen, and uses drip campaigns and autoresponders to make that happen.
3. Merge tags. Elastic Email’s merge tags help you personalize communication with clients in all occasions and for any type of content you wish to distribute. They also ensure that your dynamic content will attract immediate attention, and inspire leads to learn more about your service.
4. A/X Campaign testing. Elastic Email enables you to perform A/X tests on your campaigns in order to determine which one can reach your engagement goals. You can also test separate elements such as layouts, templates, subject lines, or posts.
5. Tracking, monitoring, and sharing performance insights. Elastic Email gives you an array of smart analytic tools to examine your delivery results. All stats and reports can be exported to a different system, or shared among the team with a single email login.

11. Marketo

A list like this would not be complete without Marketo, one of the community’s most preferred end-to-end marketing automation systems. The experienced, well-reputed, and universally applied system has much at stake for concerned marketers, including lead management, consumer marketing, mobile marketing, the unique customer base marketing, and of course – email marketing. In this regard, Marketo gives you a single, unified view of each customer, and allows you to respond to him based on what he did or didn’t do. The highly personalized contact methodology is further enhanced with rich on-scale segments, drag-and-drop maneuvers for absolute conversation control, interaction triggers, and automated A/B testing for each of your campaigns.

What is unique about Marketo?

1. Forward-to-Friend functionality. The Forward-to-Friend functionality is a unique set of social components you can use to install and distribute Marketo emails on different platforms. This means that you will be able to maintain contact with customers regardless of where they’re trying to reach you.
2. Email testing. In order to determine which content works the best for your needs, Marketo lets you perform email testing, and points out quick and dynamic changes you should consider.
3. Instant creation of landing pages. Creating landing pages with Marketo is a breeze, thanks to its drag and drop interface and its massive collection of pre-designed templates. Data capture forms can help users to aggressively profile potential contacts, which can result to more lead data over time. Landing pages can also be converted into Facebook pages, which can be tweaked and customized using Marketo’s dynamic content capabilities. 
4. Professional lead management. Lead management is also a strong suit for Marketo, which offers users the capability to gather strong leads based on pre-determined parameters and criteria such as demographics, input from sales reports, product scores, and prospect behavior.
5. Social marketing. From a social marketing point of view, Marketo will be useful in three different areas: social sharing, social engagement, and social promotion. This means that you can engage users on various leading media, including Facebook and Twitter.

12. Autopilot

Autopilot‘s developers like to describe their app as the one that makes customer journey management as simple as drawing on a whiteboard. This is because they believe being able to target the right customers in the right time and with the right message, and clients are more than happy to confirm that. For them, Autopilot has been the system that generates a doubled number of leads after only 2 weeks of on-point contact, and that genuinely transforms the generic and dull content people are usually frustrated by. At the same time, Autopilot offers several unique features in this industry, including offline and online messaging, automated lead capture across social media and CRM system, and automated data syncing on your analytic/BI dashboard.

What is unique about Autopilot?

1. Complete control over the customer’s journey. What Autopilot is best known for is the creation of exciting marketing journeys which help acquire new customers and improving your service. Various features are in place to help you show customers how to get started, subscribe, and find what they need.
2. Personalized communication. Autopilot gathers the necessary customer information to help you target leads on the right channels. Hence, you will not only be able to distribute personalized emails, but also send in-app messages, SMSs, and postcards.
3. Automated lead capturing. Autopilot will automatically turn random visitors into subscribers, and capture forms with data directly on your website, blog, and app. Consequently, each visitor will receive a tracking code, and get listed in your customer database.
4. A richer view on your customers. Autopilot integrates with tools such as Zapier, Segment, Slack, Salesforce, and more to help you build rich and robust contact visuals. It also organizes and centralizes and organizes all contact portfolios so that you won’t have to spend hours building a recipient list for your emails.
5. Smart segmenting. Unlike traditional marketing tools and widgets, Autopilot will automatically look for audiences that match a particular combination of factors, behavior, field values, URLs, and lists. In such way, it will engage them and trigger their values without you being involved in the process.

13. Zoho Campaigns

If looking to streamline and optimize your email and social media campaigns, you should give Zoho Campaigns a detailed look. Small and medium business are captivated by the tool’s effortless email marketing, automated responders, and professional email newsletters, as well as the possibility to track and report on their email activity, and promote pretty much the same attractive designs on social media. Working in synergy with other Zoho products, Campaigns soon becomes a centralized hub for all contact information which is both able to guarantee deliverability, and very well priced to meet the needs of all businesses. You should also consider it because of the free starter plan that allows you to manage even 2,000 contacts, and send over 12,000 emails per month. Awesome, isn’t it?

What is unique about Zoho Campaigns?

1. Email marketing + Social media monitoring. With Zoho Campaigns, you will get a single, affordable tool that combines email and social marketing features, and get to manage your contact engagement from a single platform.
2. Professional email newsletters. Zoho Campaigns offers you a list of beautiful default email newsletters to choose from, but also allows you to design ones of your own to engage customers in a more personalized manner.
3. Facebook-friendly system. You can simultaneously share all your campaigns on social sites, set up follow-ups and chain campaigns, and even design landing pages for your Facebook audiences.
4. Free starter package. Small and medium businesses and sole marketers will be pleased to learn that Zoho Campaigns offers a free starter plan. With this plan, you can manage even 2,000 contacts, and distribute 12,000 emails and messages at absolutely no cost.
5. Integrations with other Zoho products. The Zoho software company offers a pretty large and compact productivity suite, where all systems work in synergy to save time and optimize results. Zoho Campaigns is one of these products, and it functions seamlessly with all leading Zoho solutions.

14. Benchmark

Benchmark is another email marketing system that leverages the power of simplified technology to gain on today’s minimalist and nitpicky market. Wherever your business may be situated, you can use Benchmark email to engage audiences, collect filled signup forms, and make sure that your content is captivating and well-displayed on any device. Basically, Benchmark adjusts your promotional tools to the needs of various users, and takes into consideration all possible circumstances that may affect their quality. It also reports in real time how each of your campaigns is progressing (who opened, missed, or bounced on your emails), and lets you pull out summaries in all desired formats. The same as Zoho Campaigns, Benchmark offers a free starter plan for managing up to 2,000 contacts. Subscriptions and credit card data is not required.

What is unique about Benchmark?

1. Great Signup forms. With Benchmark’s out-of-the-box signup forms and follow up responders, you will soon build a large database of loyal customers.
2. Responsive emails for all devices. Benchmark offers a very flexible and well-equipped email designer you can use to create all sorts of messages. The designs are mobile friendly, and display perfectly on all devices. Premade templates are also available.
3. Availability in different languages. Other than English, you can use Benchmark Email in a variety of European and non-European languages, including German, Italian, Portuguese, French, Spanish, Chinese, and Japanese.
4. Multiple useful integrations. In order to keep all of your contacts engaged and well-managed, Benchmark offers seamless integrations with third-party software and social networks. On this list, you will find Shopify, PayPal, Flickr, Twitter, Magento, Zapier, and many other tools.
5. A free plan. With less than 2,000 subscribers, you will get to use Benchmark Email absolutely for free, and be able to distribute as many emails as you need. The transition to a paid plan is also made painless, as the company offers a variety of SMB-friendly plans.

15. Robly

Robly is an email marketing system that uses OpenGen technology to provide as much as 50% more opens upon application, and which is suitable both for corporate and nonprofit environments. The tool doesn’t only help you create captivating campaigns, but it also resends them with a different subject line, giving the customer a second chance to examine your product/service. The persistence and time effectiveness of this system has pleased satisfied customers with 5x larger lead lists, working around simple tactics such as replacing traditional signup widgets with fully automated popup forms. In such way, Robly introduces you to the market you’re about to serve, making it possible to share your emails on social media platforms.

What is unique about Robly?

1. OpenGen technology. With its OpenGen technology, Robly ensures that your email campaigns are read by your recipients. This functionality has proven to increase open rates to as much as 50%. It initially begins with a typical marketing content sent via email. Subscribers who did not open your first email will automatically get another campaign 1-10 days later, this time with a different subject line.
2. Robly AI. More than just sending repeat email campaigns, Robly’s patent-pending artificial intelligence innovation called Robly A.I. will analyze all details about your campaigns, history, subscriber interactions, and more. This artificial intelligence determines when is the best time to send your emails, which is when your subscribers will most likely open their inbox and check their messages.
3. A/B testing. Robly also offers an A/B testing capability that helps you cover all the bases, optimize all elements of your email campaign such as name, subject line, call to action, content, and more.
4. Exit-Intent Pop Up. Robly doesn’t use traditional sign up widgets, but a one-of-a-kind Exit-Intent technology that motivates leaving users to come back and join your mailing list.
5. Mobile responsive templates. All templates you will find within Robly’s library are ready to use on mobile devices, and contain call-to-action buttons that increase in size in order to maximize conversion.

16. iContact Pro

iContact Pro is a relatively new player in the email marketing software arena, but one that didn’t take long to impose itself across the business community. Rather than focusing solely on email marketing, iContact Pro combines workflow automation, social media management, and business analytics in a mobile-friendly package you will use on a daily basis. In the role of a loyal marketing assistant, iContact will help you create effective email campaigns, be those modified templates or unique, custom designs. Then, it will test each of the campaigns to enhance your targeting strategy, and segment leads in categories based on how they’re interacting with you. iContact Pro also tracks and records other types of engagement, including social media conversations, landing pages, newsletters and releases, events, and website visits.

What is unique about iContact Pro?

1. All-in-one marketing kit. iContact Pro is a system that will automate all important marketing functions, manage and score all of your leads through the same platform. You can use it to create stunning landing pages, as well as other HTML-based and template-based pages.
2. Timely campaign management. With iContact, you can timely and easily manage your email campaigns. It offers tools for segmentation, A/B testing, and a message builder to fulfill email marketing needs of your company. It comes with list management tools for easy management of client IDs which often becomes difficult due to an ever increasing client base.
3. Social media management features. This cloud-based software also integrates social media management features and robust monitoring and analytic functionalities. The social media management tools let you schedule and post your campaigns all over Facebook, Twitter, and LinkedIn.
4. Salesforce integration. Contact can be integrated with Salesforce to help you manage the sales process. It also offers analytics which is vital for strategic planning and can be used to access the marketing funnel reports. These analytics play a major role in deciding resource allocation for marketing and sales in long term planning.
5. SMB-friendly pricing. iContact Pro is made affordable to small and medium businesses, as its prices don’t depend on the features it offers, but on the number of subscribers you wish to manage. Teams with less than 500 subscribers, for instance, can purchase it for only $11.90/month.

17. Adestra

Adestra is an 1 on 1 digital marketing platform for businesses and nonprofits that depicts unique customer patterns and trails, and adjusts accordingly your standard emails to help you make the most of each conversation. It is mostly recognizable by data driven automation (MessageFocus), namely making it possible to distribute relevant content to each and every user. It is also suitable for mobile-first marketing, and it integrates seamlessly within any technology ecosystem. The interface is clutter-free, and a genuine no-brainer for starters, but a knowledgeable and friendly support team is still in place to assist you at any time. Visual reporting is another distinctive advantage of this system.

What is unique about Adestra?

1. A quick Form Builder tool.  The Form Builder tool which comes with this software allows you to create templates and landing pages with ease to help generate leads using email marketing. The templates are pre-built and easy to embed on different locations.
2. Email triggers for your customers. The email triggers give your business the needed support regarding recovering customers who are no longer buying from your store. This can help turn abandoned carts into customers, and target inactive customers by sending custom texts or messages to them enticing them to continue buying.
3. A variety of reporting features. Adestra’s analytic kit lets you report about the activities and performance of individual marketing campaigns. It includes social sharing reporting, client reporting, and geo-tracking that allows you to know which email the client has opened. This form of monitoring ensures that you stay on top of your marketing campaign.
4. Dynmark integration. Adestra relies on its Dynmark integration to enable SMS marketing activities and distribution of short promotional texts.
5. Real-time support. Adestra’s friendly and experienced support team is accessible 24/7, and reachable on different channels (email, phone, live chat).

18. Klaviyo

Klaviyo may not have the decades of worldwide experience leading email marketing tools have, but this doesn’t mean that you shouldn’t keep it high on your priority list. A good reason to trust Klaviyo is its commitment to web eCommerce businesses, whom it helps boost sales with end-to-end email and social media optimization. More specifically, Klaviyo helps you target the best audience for your product/service, build up a catchy campaign and optimize your efforts on social media, and measure the effects of your work. Designed for automated and powerful segmentation, Klaviyo eliminates all need to hold on to messy spreadsheets, puts in action ready-to-use autoresponders,  and genuinely examines vital sales metrics. On top of it all, Klaviyo works in synergy with all tools modern marketers use, including Magento, Shopify, BigCommerce, and +CustomPlatforms.

What is unique about Klaviyo?

1. Improved segmentation. Klaviyo allows combining and excluding different segments for even more powerful targeting and lets marketers specify multiple criteria for their segments. For example, marketers can target female customers who bought a purse during the holiday season.
2. Custom web-tracking system. The custom web-tracking system uses browsing behavior and helps target people who meet specific criteria e.g. male customers who looked at t-shirts 5 times during a week. This means virtually unlimited targeting opportunities, thanks to the variety of available options.
3. Facilitated tracking of Facebook audiences. Klaviyo makes is easier for marketers to use existing segments to target Facebook Ad campaigns, while events, custom attributes and properties can be used to personalize emails.
4. WYSIWYG email editor. Klaviyo’s WYSIWYG email editor is particularly useful in creating great emails and makes the whole process significantly easier, while the responsive email templates are optimized for mobile devices and work well on all devices.
5. Connectivity. Klaviyo can be integrated easily with other popular software, including Magento, Shopify, Salesforce, ZOHO and Zendesk, while the Open API is a big plus for users who want custom integrations. Product recommendations can easily be pulled in and used with email campaigns, while dynamic data blocks are effective in obtaining and pulling in product/order information for more personalized emails.

19. FreshMail

There are many reasons to consider FreshMail as your future marketing assistant: simple, very creative, and with its top features sliced accordingly across affordable plan, FreshMail has something to offer to each and every marketer out there. It doesn’t matter what you’re promoting – the tool will be handy to distribute notably personalized mail, draft one-of-a-kind campaigns, and help you build a memorable reputation among your customers. For the purpose, FreshMail uses features such as the intuitive template editor, fast and unrestricted data access, and collaboration widgets for remote teams. Our experts also suggest it because of the largest variety of premade templates that cater to every industry, and which will most certainly save busy marketers a solid amount of time.

What is unique about FreshMail?

1. Downloads and distributes your HTML newsletters. FreshMail works best for businesses that have a CMS able to create newsletters in HTML as it can download and send them with just a single click. It also allows easy management of promotions with bar codes and can generate promotion & discount barcodes, which the subscribers can use in retail stores. These codes are created automatically, and sent as email attachments.
2. Editing any HTML file. FreshMail will not only download and save HTML files as such, but also allow you to edit them. Each of the files can then be used as a template for newsletters. In addition, emails can also be created with an image turn-off-state as half of the subscribers keep images turned off.
3. Code-free template editor. Each of the templates offered in FreshMail’s library can be transformed into an eye-catching newsletter, and won’t require any coding experience for the purpose.
4. Clever autoresponders. FreshMail can auto-respond to customer’s actions and automate communication with subscribers through its ‘clever autoresponders’. Meanwhile, its real-time email tracking allows easily finding out who is interested in the newsletters and following its links.
5. Inbox Inspector. The Inbox Inspector allows marketers to see how their marketing messages will look in 24 different email clients, while the Spam Test frees the emails from content which may get treated as spam. Automatic A/B testing works well to test alternatives and helps find out what really works.

20. sendinblue

Last, but not least, we have sendinblue, another hassle-free marketing automation tool that adds value to your engagement efforts. Hassle-free and intuitive, sendinblue turns your email and SMS campaigns into active conversion triggers, and gives you unique possibilities such as delivering transactional emails, automating custom sales workflows, and optimizing performance via throughout reporting. The system is easy to use and even easier to connect with third party systems (you get a profy set of open API), and is currently available in several languages to make engagement easier. Satisfied users report up to 70% more opened emails since they started using it, which should be an additional motive to give the tool a chance.

What is unique sendinblue?

1. Unified marketing campaigns. With sendinblue Email, you can launch effective marketing campaigns with no boundaries. Easily manage your company’s marketing campaigns, SMS messages, and transactional emails in one powerful and simple platform.
2. Powerful contact management. Contact management is important to any successful email marketing campaigns. That is why sendinblue gives you the tools you need to handle your contacts effectively and easily: importing contacts, categorizing lists, behavioral filters, collection forms, and more.
3. A variety of independent modules. With sendinblue it is you that chooses which features you want to use. In order to ensure the success of your specific marketing campaigns, sendinblue lets you combine any of the following independent modules: marketing campaigns, transactional text messages, transactional emails, newsletters, and more.
4. Order tracking. One of sendinblue’s best known monitoring capabilities is that it tracks all of your orders in real time. The tool can also be integrated with Google Analytics for in-depth examination of your campaigns.
5. A variety of pricing plans. In order to let all users find a package they like and can afford, sendinblue offers multiple different plans between $7.37 and $1,946 per month. A free plan for small teams and sole marketers is also available.

Category: B2B News, Featured Articles

The Red Sox responded in kind on Tuesday, filing a complaint against the Yankees claiming that the team uses a camera from its YES television network exclusively to steal signs during games, an assertion the Yankees denied.

It is unclear what penalties, if any, Commissioner Rob Manfred will issue against the Red Sox and whether he will order a more expansive investigation to determine the extent of the Red Sox’ sign-stealing system. It is also unclear how he will proceed with the countercomplaint.

“We will conduct a thorough investigation on both sides,” Mr. Manfred said to reporters at Fenway Park, where he was present for an unrelated event. “We’re 100 percent comfortable that it is not an ongoing issue.”

Mr. Manfred said he believed he had the power to punish teams in connection with such cheating.

“Could it happen? You know, is there the authority to do that? I think the answer to that, under the major league constitution, is yes,” he said. “Has it ever happened with this type of allegation? I think the answer is — I know the answer is no.

“And the reason for that,” he added, “is it’s just very hard to know what the actual impact on any particular game was of an alleged violation.”

Boston’s manager, John Farrell, said he was aware that the players were trying to steal signs but said that he did not know they were using electronics.

“I’m aware of the rule,” Mr. Farrell said. “Electronic devices are not to be used in the dugout. Beyond that, all I can say is it’s a league matter at this point.”

Stealing signs is believed to be particularly effective when there is a runner on second base who can both watch what hand signals the catcher is using to communicate with the pitcher and can easily relay to the batter any clues about what type of pitch may be coming. Such tactics are allowed as long as teams do not use any methods beyond their eyes. Binoculars and electronic devices are both prohibited.

In recent years, as cameras have proliferated in major league ballparks, teams have begun using the abundance of video to help them discern opponents’ signs, including the catcher’s signals to the pitcher. Some clubs have had clubhouse attendants quickly relay information to the dugout from the personnel monitoring video feeds.

But such information has to be rushed to the dugout on foot so it can be relayed to players on the field — a runner on second, the batter at the plate — while the information is still relevant. The Red Sox admitted to league investigators that they were able to significantly shorten this communications chain by using electronics. In what mimicked the rhythm of a double play, the information would rapidly go from video personnel to a trainer to the players.

As part of the inquiry, baseball investigators have interviewed the Red Sox team trainers and outfielder Chris Young, a former Yankees player. The Red Sox told league investigators that Mr. Farrell; Boston’s president for baseball operations, Dave Dombrowski; and other front-office officials were not aware of the sign-stealing operation, the people said.

In the first game of the August series in question, the Red Sox prospered the first time they put a runner on second. It occurred in the second inning, and Rafael Devers promptly hit a home run, giving the Red Sox a 2-0 lead. The Red Sox went 5 for 8 in that game when they had a man on second.

Photo

Their success when they had a runner on second in the other two games of the series was mixed: 1 for 6 in the second contest and 3 for 10 in the third.

The video provided to the commissioner’s office by the Yankees was captured during the first two games of the series and included at least three clips. In the clips, the Red Sox assistant athletic trainer, Jon Jochim, is seen looking at his Apple Watch and then passing information to outfielder Brock Holt and second baseman Dustin Pedroia, who was injured at the time but in uniform. In one instance, Pedroia is then seen passing the information to Young.

The Red Sox’ tactics will add to the longstanding rivalry with the Yankees. Outside New England, the case will be reminiscent of the Spygate controversy that erupted a decade ago when the world-beating Patriots were found to have violated N.F.L. rules by spying on opponents to gain an edge.

Mr. Manfred is in a difficult position as he decides how to discipline the team and whether to continue investigating.

In Spygate, the N.F.L. commissioner, Roger Goodell, hastily took away a first-round draft pick and fined the Patriots and their coach, Bill Belichick, before conducting a thorough investigation. When more evidence of cheating later emerged, Goodell was accused of trying to minimize the damage and protect one of the sport’s premier franchises.

In baseball, the most infamous incident involving sign stealing played out in 1951, when the New York Giants overcame a 13 ½-game deficit over the final two months of the season to catch the Brooklyn Dodgers. The Giants went on to beat the Dodgers in a playoff for the pennant.

Fifty years later, The Wall Street Journal revealed that the Giants had spies at the Polo Grounds who used a telescope to steal signs from the opposing catcher, which were then relayed to Giants players from the bullpen.

In 1997, teams accused the Mets of planting small cameras near home plate in Shea Stadium to steal signals. The Mets denied that they had used the cameras to try to do so, and the league did not take any action.

More recently, the Philadelphia Phillies faced sign-stealing accusations in 2011. Several teams logged complaints with the commissioner’s office that the team used binoculars and other unauthorized methods to steal signs. Major League Baseball never imposed sanctions on the Phillies.

Continue reading the main story

Professor Krankenstein was the most influential genetic engineer of his time.

When, in the spring of 2030, he almost incidentally invented the most terrible biological weapon known to humanity it took him about three seconds to realize that should his invention fall into the hands of one of the superpowers -- or into the hands of any common idiot, really -- it could well mean the end of the human race.

He wasted no time. He destroyed all the artifacts in the lab. He burned all the notes and hard disks of all the computers they’ve used in the project. He seeded false information all over the place to lead future investigators off the track.

Now, left with the last remaining copy of the doomsgerm recipe, he was contemplating whether to destroy it.

Yes, destroying it would keep the world safe. But if such a breakthrough in genetic engineering was used in a different way it could have solved the hunger problem by producing enough artificial food to feed the swelling population of Earth. And if global warming went catastrophic, it could have been used to engineer microorganisms super-efficient at sequestering carbon dioxide and methane from atmosphere.
In the end he decided not to destroy but rather to encrypt the recipe, put it into a tungsten box, encase the box in concrete and drop it from a cruise ship into Mariana Trench.

The story would have ended there if it was not for one Hendrik Koppel, a rather simple-minded person whom professor Krankenstein hired to help him to move the tungsten-concrete box around. Professor didn’t even met him before he destroyed all his doomsgerm research. Still, Hendrik somehow realized that the issue was of interest to superpowers (Was professor Krankenstein sleep-talking?) and sold the information about the location of the box to several governments.

By the beginning of October the news hit that an American aircraft carrier is heading in the direction of Mariana Trench.

Apparently, there was also a Russian nuclear submarine on its way to the same location.

Chinese government have sent a fleet of smaller, more versatile, oceanographic vessels.

After the initial bout of despair, professor Krankenstein realised that with his superior knowledge of the position of the box he could possibly get to the location first and destroy the box using an underwater bomb.

He used his life savings to buy a rusty old ship called Amor Patrio, manned it with his closest collaborators and set up for Pacific Ocean.

...

Things haven't gone well. News reported that Americans and Chinese were approaching the area while Amor Patrio's engine broke and the crew was working around the clock to fix it.

Finally, they fixed it and approached Mariana Trench.

It was at that point that the news reached them: The box was found by Russians and transported to Moscow. It was now stored in the vault underneath KGB headquarters. There was a whole division of spetsnaz guarding the building. The building itself was filled with special agents, each trained in twelve ways of silently killing a person.

Professor Krankenstein and his associated held a meeting on board of Amor Patrio, in the middle of Pacific Ocean. People came up with desperate proposals: Let's dig a tunnel underneath Moscow river. Let's blackmail Russians by re-creating the virus and threatening them to disperse it in Russia. Nuke the entire Moskovskaya Oblast! There was no end to wild and desperate proposals.

Once the stream of proposals dried up, everyone looked at professor Krankenstein, awaiting his decision.

The silence is almost palpable.

Professor Krankenstein slowly took out his iconic pipe and lighted it with the paper which had the decryption key written on it.

No full story yet, but let's assume a king is at a quest. At some point he realizes that a small item, say a specific hairpin, is needed to complete the quest. He clearly remembers he used to own the hairpin, but he has no idea whether it's still in his possession and if so, where exactly it is. He sends a messenger home asking his counsellors to look for the hairpin and let him know whether they've found it or not.

King's enemies need that information as well so the next day, when the messager is returning, they ambush him and take the message. Unfortunately, the message is encrypted. The messager himself knows nothing about the pin.

Many experienced cryptographers are working around the clock for days in row to decrypt the message but to no avail.

Finally, a kid wanders into the war room. She asks about what they are doing and after some thinking she says: "I know nothing about the high art of cryptography and in no way I can compare to esteemed savants in this room. What I know, though, is that King's pallace has ten thousand rooms, each full of luxury, pictures and finely carved furniture. To find a hairpin in such a place can take weeks if not months. If there was no hairpin it would take at least that long before they could send the messenger back with negative reply. So, if the messager was captured on his way back on the very next day, it can mean only a single thing: The hairpin was found and your encrypted message says so."

Here Legrand, having re-heated the parchment, submitted It my inspection. The following characters were rudely traced, in a red tint, between the death's-head and the goat:

53++!305))6*;4826)4+.)4+);806*;48!8`60))85;]8*:+*8!83(88)5*!;
46(;88*96*?;8)*+(;485);5*!2:*+(;4956*2(5*-4)8`8*; 4069285);)6
!8)4++;1(+9;48081;8:8+1;48!85;4)485!528806*81(+9;48;(88;4(+?3
4;48)4+;161;:188;+?;

"But," said I, returning him the slip, "I am as much in the dark as ever. Were all the jewels of Golconda awaiting me on my solution of this enigma, I am quite sure that I should be unable to earn them."

"And yet," said Legrand, "the solution is by no means so difficult as you might be led to imagine from the first hasty inspection of the characters. These characters, as any one might readily guess, form a cipher --that is to say, they convey a meaning; but then, from what is known of Kidd, I could not suppose him capable of constructing any of the more abstruse cryptographs. I made up my mind, at once, that this was of a simple species --such, however, as would appear, to the crude intellect of the sailor, absolutely insoluble without the key."

"And you really solved it?"

"Readily; I have solved others of an abstruseness ten thousand times greater. Circumstances, and a certain bias of mind, have led me to take interest in such riddles, and it may well be doubted whether human ingenuity can construct an enigma of the kind which human ingenuity may not, by proper application, resolve. In fact, having once established connected and legible characters, I scarcely gave a thought to the mere difficulty of developing their import.

"In the present case --indeed in all cases of secret writing --the first question regards the language of the cipher; for the principles of solution, so far, especially, as the more simple ciphers are concerned, depend on, and are varied by, the genius of the particular idiom. In general, there is no alternative but experiment (directed by probabilities) of every tongue known to him who attempts the solution, until the true one be attained. But, with the cipher now before us, all difficulty is removed by the signature. The pun on the word 'Kidd' is appreciable in no other language than the English. But for this consideration I should have begun my attempts with the Spanish and French, as the tongues in which a secret of this kind would most naturally have been written by a pirate of the Spanish main. As it was, I assumed the cryptograph to be English.

"You observe there are no divisions between the words. Had there been divisions, the task would have been comparatively easy. In such case I should have commenced with a collation and analysis of the shorter words, and, had a word of a single letter occurred, as is most likely, (a or I, for example,) I should have considered the solution as assured. But, there being no division, my first step was to ascertain the predominant letters, as well as the least frequent. Counting all, I constructed a table, thus:

Of the character 8 there are 33.

   ;     "     26.
   4     "     19.
 + )     "     16.
   *     "     13.
   5     "     12.
   6     "     11.
 ! 1     "      8.
   0     "      6.
 9 2     "      5.
 : 3     "      4.
   ?     "      3.
   `     "      2.
 - .     "      1.

"Now, in English, the letter which most frequently occurs is e. Afterwards, the succession runs thus: a o i d h n r s t u y c f g l m w b k p q x z. E however predominates so remarkably that an individual sentence of any length is rarely seen, in which it is not the prevailing character.

"Here, then, we have, in the very beginning, the groundwork for something more than a mere guess. The general use which may be made of the table is obvious --but, in this particular cipher, we shall only very partially require its aid. As our predominant character is 8, we will commence by assuming it as the e of the natural alphabet. To verify the supposition, let us observe if the 8 be seen often in couples --for e is doubled with great frequency in English --in such words, for example, as 'meet,' 'fleet,' 'speed, 'seen,' 'been,' 'agree,' &c. In the present instance we see it doubled less than five times, although the cryptograph is brief.

"Let us assume 8, then, as e. Now, of all words in the language, 'the' is the most usual; let us see, therefore, whether they are not repetitions of any three characters in the same order of collocation, the last of them being 8. If we discover repetitions of such letters, so arranged, they will most probably represent the word 'the.' On inspection, we find no less than seven such arrangements, the characters being ;48. We may, therefore, assume that the semicolon represents t, that 4 represents h, and that 8 represents e --the last being now well confirmed. Thus a great step has been taken.

"But, having established a single word, we are enabled to establish a vastly important point; that is to say, several commencements and terminations of other words. Let us refer, for example, to the last instance but one, in which the combination ;48 occurs --not far from the end of the cipher. We know that the semicolon immediately ensuing is the commencement of a word, and, of the six characters succeeding this 'the,' we are cognizant of no less than five. Let us set these characters down, thus, by the letters we know them to represent, leaving a space for the unknown--

t eeth.

"Here we are enabled, at once, to discard the 'th,' as forming no portion of the word commencing with the first t; since, by experiment of the entire alphabet for a letter adapted to the vacancy we perceive that no word can be formed of which this th can be a part. We are thus narrowed into

t ee,

and, going through the alphabet, if necessary, as before, we arrive at the word 'tree,' as the sole possible reading. We thus gain another letter, r, represented by (, with the words 'the tree' in juxtaposition.

"Looking beyond these words, for a short distance, we again see the combination ;48, and employ it by way of termination to what immediately precedes. We have thus this arrangement:

the tree ;4(+?34 the,

or substituting the natural letters, where known, it reads thus:

the tree thr+?3h the.

"Now, if, in place of the unknown characters, we leave blank spaces, or substitute dots, we read thus:

the tree thr...h the,

when the word 'through' makes itself evident at once. But this discovery gives us three new letters, o, u and g, represented by + ? and 3.

"Looking now, narrowly, through the cipher for combinations of known characters, we find, not very far from the beginning, this arrangement,
83(88, or egree, which, plainly, is the conclusion of the word 'degree,' and gives us another letter, d, represented by !.

"Four letters beyond the word 'degree,' we perceive the combination

46(;88*.

"Translating the known characters, and representing the unknown by dots, as before, we read thus:

th.rtee.

an arrangement immediately suggestive of the word 'thirteen,' and again furnishing us with two new characters, i and n, represented by 6 and *.

"Referring, now, to the beginning of the cryptograph, we find the combination,

53++!.

"Translating, as before, we obtain

.good,

which assures us that the first letter is A, and that the first two words are 'A good.'

"To avoid confusion, it is now time that we arrange our key, as far as discovered, in a tabular form. It will stand thus:

5 represents a
!    "       d
8    "       e
3    "       g
4    "       h
6    "       i
*    "       n
+    "       o
(    "       r
;    "       t

"We have, therefore, no less than ten of the most important letters represented, and it will be unnecessary to proceed with the details of the solution. I have said enough to convince you that ciphers of this nature are readily soluble, and to give you some insight into the rationale of their development. But be assured that the specimen before us appertains to the very simplest species of cryptograph. It now only remains to give you the full translation of the characters upon the parchment, as unriddled. Here it is:

'A good glass in the bishop's hostel in the devil's seat twenty-one degrees and thirteen minutes northeast and by north main branch seventh limb east side shoot from the left eye of the death's-head a bee line from the tree through the shot fifty feet out.'"

"But," said I, "the enigma seems still in as bad a condition as ever. How is it possible to extort a meaning from all this jargon about 'devil's seats,' 'death's-heads,' and 'bishop's hostel'?"

"I confess," replied Legrand, "that the matter still wears a serious aspect, when regarded with a casual glance. My first endeavor was to divide the sentence into the natural division intended by the cryptographist."

"You mean, to punctuate it?"

"Something of that kind."

"But how was it possible to effect this?"

"I reflected that it had been a point with the writer to run his words together without division, so as to increase the difficulty of solution. Now, a not overacute man, in pursuing such an object, would be nearly certain to overdo the matter. When, in the course of his composition, he arrived at a break in his subject which would naturally require a pause, or a point, he would be exceedingly apt to run his characters, at this place, more than usually close together. If you will observe the MS., in the present instance, you will easily detect five such cases of unusual crowding. Acting on this hint, I made the division thus:

'A good glass in the bishop's hostel in the devil's --twenty-one degrees and thirteen minutes --northeast and by north --main branch seventh limb east side --shoot from the left eye of the death's-head --a bee-line from the tree through the shot fifty feet out.'"

"Even this division," said I, "leaves me still in the dark."

A portal suddenly opened on the starboard ejecting a fleet of imperial pursuit vessels. The propulsion system of my ship got hit before the shield activated. I’ve tried to switch on the backup drive but before it charged to 5% I was already dangling off a dozen tractor beams.

It wasn’t much of a fight. They’ve just came and picked me up as one would pick up a box of frozen strawberries in a supermarket.

I must have passed out because of pressure loss because the next thing I remember is being in a plain white room with my hands cuffed behind my back.

There was a sound of door opening and a person walked into my field of vision.

It took me few seconds to realize who the man was. He was wearing an old-fashioned black suit and a bowler hat, black umbrella in his hand, not the baggy trousers seen on his official portraits. But then he smiled and showed the glistening golden teeth on the left side and his own healthy camel-like teeth on the right and the realization hit me.

It was him. Beylerbey Qgdzzxoglu in person.

“Peace be upon you,” he said. Then he sat down on the other side of a little coffee table, made himself comfortable and put his umbrella on the floor.

“We have a little matter to discuss, you and I,” he said.

He took a paper out of his pocket and put in on the coffee table, spinning it so that I can read it.

“Attack the Phlesmus Pashalik,” said one line.

“Attack the Iconium Cluster,” said the line below it.

The rest of the sheet was empty except for holographic seal of High Command of Proximian Insurgency.

"Comandante Ribeira is no fool," he said, "And this scrap of paper is not going to convince me that he's going to split his forces and attack both those places at the same time. Our strategic machines are vastly more powerful than Proximian ones, they've been running hot for the past week and out lab rats tell us that there's no way to win that way."

"You are right, O leader of men," I said.

I knew that this kind of empty flattery was used at the Sublime Porte but I was not sure whether it wasn't reserved for the sultan alone.

Qgdzzxoglu smiled snarkily but haven't said anything. Maybe I was going to live in the end.

"I have no loyalty for the Proximian cause and before the rebelion I have lived happily and had no thoughts of betrayal. And now, hoping for your mercy, I am going to disclose the true meaning of this message to you."

"It is a code, O you, whose slipper weights heavily upon the neck of nations," I said, "The recepient is supposed to ignore the first sentence and only follow the second one."

I hoped I haven't overdone it. Being honest with the enemy is hard.

"So you are trying to convince me that de Ribeira is going to attack Iconium," he gave me a sharp look, apparently trying to determine whether I was lying or not. "And you know what? We've got our reports. And our reports are saying that rebels will try to trick us into moving all our forces into Iconium and then attack pashalik of Phlesmus while it's undefended. And if that's what you are trying to do bad things are going to happen to your proboscis."

"The Most Holy Cross, John XXIII and Our Lady of Africa have already got a command to move to Iconium cluster. And you should expect at least comparable fire force from elsewhere."

...

The messenger has no intention to suffer to win someone else's war. At the same time it's clear that if he continues to tell truth he will be tortured. So he says that general is right and it's the first sentence that should be taken into account.

The general begins to feel a bit uneasy at this point. He has two contradictory confessions and it's not at all clear which one is correct.

He orders the torture to proceed only to make the messager change his confession once again.

...

“Today, the God, the Compassionate, the Merciful have taught me that there are secrets that cannot be given away. You cannot give them away to save yourself from torture. You cannot give them away to save your kids from being sold to slavery. You cannot give them away to prevent the end of the world. You just cannot give them away and whether you want to or not matters little.”

Here's a simple game for kids that shows how asymmetric encryption works in principle, makes the fact that with only public key at your disposal encryption may be easy while decryption may be so hard as to be basically impossible, intuitive and gives everyone a hands-on experience with a simple asymmetric encryption system.

Here's how it works:

Buy a dictionary of some exotic language. The language being exotic makes it improbable that any of the kids involved in the game would understand it. Also, it makes cheating by using Google Translate impossible.

Let's say you've opted for Eskimo language. The story of the game can be located at the North Pole after all.

You should prefer a dictionary that comes in two bands: English-Eskimo dictionary and Eskimo-English dictionary. The former will play the role of public key and the latter the role of secret key. Obviously, if there's no two-band dictionary available, you'll have to cut a single-band one in two.

To distribute the public key to everyone involved in the game you can either buy multiple copies of English-Eskimo dictionary, which may be expensive, or you can simply place a single copy at a well-known location. In school library, at a local mom-and-pop shop or at a secret place known only to the game participants.

If a kid wants to send an encrypted message to the owner of the secret key, they just use the public key (English-Eskimo dictionary) to translate the message, word-by-word, from English to Eskimo. The owner of the secret key (Eskimo-English dictionary) can then easily decrypt the message by translating it back into English.

However, if the message gets intercepted by any other game participant, decrypting it would be an extremely time consuming activity. Each word of the message would have to be found in English-Eskimo dictionary, which would in turn mean scanning the whole dictionary in a page-by-page and word-by-word manner!

It's a puppet show. There are two hills on the stage with country border between them. Law-abiding citizen is on the right hill. Smuggler enters the stage on the left.

SMUGGLER: Hey, you!

CITIZEN: Who? Me?

SMUGGLER: Do you like booze?

CITIZEN: Sure I do. And who are you?

SMUGGLER: I'm the person who will sell you some booze.

CITIZEN: What about cigarettes?

SMUGGLERS: Sure thing. Cheap Ukrainian variety for $1 a pack. Also Slovenian Mariboro brand.

CITIZEN: Thanks God! I am getting sick of our government trying to make me healthy!

Border patrol emerges from a bush in the middle of the stage.

PATROL: Forget about it, guys! This is a state border. Nothing's gonna pass one way or the other. You better pack your stuff and go home.

SMUGGLER: Ignore him. We'll meet later on at some other place, without border patrol around, and you'll get all the booze and cigarettes you want.

PATROL: Ha! I would like to see that. Both of you are going to end up in jail.

CITIZEN: He's right. If you tell me where to meet, he's going to hear that, go there and arrest us.

...

Smuggler has a list of possible places to meet:

1. Big oak at 57th mile of the border.
   
2. Lower end of Abe Smith's pasture.
   
3. ...
   
4. ...

He obfuscates each entry and shout them to the citizen in no particular order.

Citizen chooses one of the puzzles and de-obfuscates it. It takes him 10 minutes. The de-obfuscates message reads: "18. Behind the old brick factory."

CITIZEN (cries): Eighteen!

SMUGGLER: Ok, got it, let's meet there in an hour!

PATROL: Oh my, oh my. I am much better at de-obfuscation than that moron citizen. I've already got two messages solved. But one has number 56, the other number 110. I have no idea which one is going to be number 18. There's no way I can find the right one in just one hour!

The curtain comes down. Happy gulping sounds can be heard from the backstage.

Mr. X is approached in the subway by a guy who claims to be an alien stranded on Earth and to possess time machine that allows him to know the future. He needs funds to fix his flying saucer but filling in winning numbers for next week's lottery would create a time paradox. Therefore, he's willing to sell next week's winning numbers to Mr. X at a favourable price.

Mr. X, as gullible as he is, feels that this may be a scam and asks for a proof. Alien gives him the next week's winning numbers in encrypted form so that Mr. X can't use them and then decide not to pay for them. After the lottery draw he'll give Mr. X the key to unlock the file and Mr. X can verify that the prediction was correct.

After the draw, Mr. X gets the key and lo and behold, the numbers are correct! To rule out the possibility that it happened by chance, they do the experiment twice. Then thrice.

Finally, Mr. X is persuaded. He pays the alien and gets the set of numbers for the next week's draw.

But the numbers drawn are completely different.

And now the question: How did the scam work?

NOTE: The claim about the time paradox is super weak. To improve the story the alien can ask for something non-monetary (sex, political influence). Or, more generally, positive demonstration of knowledge of the future can be used to make people do what you want. E.g. "I know that an asteroid is going to destroy Earth in one year. Give me all your money to build a starship to save you."

Count the number of hours you sit each day. Be honest.

"If you commute an hour in the morning and hour after work — that's two hours, and if you sit at an eight-hour-a-day desk job that's 10," says epidemiologist Loretta DiPietro of the Milken Institute School of Public Health at George Washington University.

"Then you come home at, say, 6 p.m., eat dinner and crash into your recliner for another three to four hours," says DiPietro. "That's 13 to 14 hours of sitting."

Being immobile like that for many hours each day does more than raise the risk of a host of diseases. DiPietro and her colleagues have good evidence that, as the years wear on, it actually reduces the ability of older people to get around on foot at all.

In a study of sitting and walking ability that surveyed people ages 50 to 71 across 8 to 10 years, those who tended to sit the most and move the least had more than three times the risk of difficulty walking by the end of the study, when compared to their more active counterparts.

Some ended up unable to walk at all. The study appears in the current issue of The Journals of Gerontology: Medical Sciences.

Prolonged sitting and TV watching were particularly harmful, DiPietro found, especially when combined with low levels of total physical activity. Young bodies may rebound from prolonged sitting with an hour at the gym, she says. But that seems less true in late middle age.

"Sitting and watching TV for long periods, especially in the evening," she says, "has got to be one of the most dangerous things that older people can do." And the period studied — the mid-1990s to 2005, or so — was even before the advent of rampant online streaming of shows, she notes. The problem today is likely even worse.

"Before binge watching, at least when a show ended you got up and walked around," DiPietro says. "It's now possible to watch several hours without moving."

Though being sedentary at work is also a risk, office employees tend to at least get up now and then, walk down the hall to the printer or restroom, and go to lunch, she says. Or at least workers used to do that. Increasingly, she says, many of us of all ages are engineering much of that light activity out of our lives.

"We now use the Internet to go shopping, order groceries, send messages, and even gossip," DiPietro says. "We used to walk down the hall and gossip; now we send it via email or text."

To measure the effect of prolonged sitting on mobility, DiPietro and colleagues took data from the large NIH-AARP Diet and Health Study of men and women ages 50 to 71. The participants were all healthy when the study started in 1995 and 1996.

The researchers recorded how much those in the study watched TV, exercised or did gardening, housework or other physical activity at the beginning of the investigation. They included "light" physical activity like "puttering around, walking to get the mail, or walking to the car" says DiPietro.

The results: Those who watched five or more hours of TV per day had a 65 percent greater risk of reporting a mobility disability at the study's end, compared with those who watched less than two hours per day. DiPietro says this association was independent of their level of total physical activity and other factors known to affect the ability to easily move around.

She offers an antidote: Get up at least every 30 minutes when staring at a screen.

"And if you insist on staying seated during that 15 second interval between episodes," DiPietro says, "at least stand up, march in place, jump around, kick legs — do anything to move about for at least one to two minutes."

The result of that would be "phenomenal," to mobility, she says, and be at least a start toward heart health, too.

Dr. Andrew Freeman, who directs cardiovascular prevention and wellness at National Jewish Health in Denver, and represents the American College of Cardiology, says people should do even more higher intensity exercise regularly — at least to the point of being "breathless."

But that doesn't necessarily mean jogging around the neighborhood, he says. It can be as simple as walking at a slow pace.

In an older population, Freeman says if you're going to be sedentary, you should try to be "as active" as you can when not sedentary. That may sound like common sense, he says, but the findings of the DiPietro study underscore the importance. Just five minutes a day of brisk movement, he says, is beneficial.

For Denver businesswoman Liz King, that translates to a 20-minute daily walk during the week, and longer on the weekends. King says she's joined walking groups — including the national program Walk With A Doc— as a way to build more activity into her day.

King is 61 years old and very busy starting a vegan food company in Denver — but does it, pretty much, sitting down, she says.

"When you have your own business, you have a little more flexibility," she says. "But I'll tell you one thing that's constant — that's eight hours, at least, of looking at the screen and sitting in the chair."

And, for her, being glued to the computer — and the chair — doesn't necessarily stop when the work day ends.

"The inbox that may be overflowing," she says. Or she's doing homework for an online course. She'll check in with a chat session or with other family and friends online.

Then there's about an hour or more of watching television, King says. It all adds up.

So she makes sure to get out and walk at least a little bit, every day. She varies the pace, she admits, and laughs.

"If I'm walking solo I'm probably walking at a more leisurely pace," she says, "because I tend to take a picture of the occasional wildflower, or the clouds that are in a wonderful formation."

But on a speedier walk with a group, she keeps up.

It's all a big win for King's health, Freeman says, because she's away from the screen and moving. Exercise, he says, is nature's best medicine.

A proposal by Calder Hansen

Why does Mars need a flag?

What does this flag represent?

Why use this flag instead of another that already exists?

---

---

​Would a flag actually work on Mars?

Specifications

Image files

​If you know someone who would be interested in this flag, please send them the link!

DocumentationDownloadsBrochure

When getting started with wolkenkit, you may be interested in the theoretical background of how wolkenkit works. That's why we created a brochure that introduces you to CQRS, domain-driven design, and event-sourcing in a concise form.

The brochure is 68 pages, DIN A6 portrait (148mm x 105mm).

Download

Order

For professionally printed copies contact the native web.

Uber Engineering is committed to developing technologies that create seamless, impactful experiences for our customers. We are increasingly investing in artificial intelligence (AI) and machine learning (ML) to fulfill this vision. At Uber, our contribution to this space is Michelangelo, an internal ML-as-a-service platform that democratizes machine learning and makes scaling AI to meet the needs of business as easy as requesting a ride.

Michelangelo enables internal teams to seamlessly build, deploy, and operate machine learning solutions at Uber’s scale. It is designed to cover the end-to-end ML workflow: manage data, train, evaluate, and deploy models, make predictions, and monitor predictions. The system also supports traditional ML models, time series forecasting, and deep learning.

Michelangelo has been serving production use cases at Uber for about a year and has become the de-facto system for machine learning for our engineers and data scientists, with dozens of teams building and deploying models. In fact, it is deployed across several Uber datacenters, leverages specialized hardware, and serves predictions for the highest loaded online services at the company.

In this article, we introduce Michelangelo, discuss product use cases, and walk through the workflow of this powerful new ML-as-a-service system.

Motivation behind Michelangelo

Before Michelangelo, we faced a number of challenges with building and deploying machine learning models at Uber related to the size and scale of our operations. While data scientists were using a wide variety of tools to create predictive models (R, scikit-learn, custom algorithms, etc.), separate engineering teams were also building bespoke one-off systems to use these models in production. As a result, the impact of ML at Uber was limited to what a few data scientists and engineers could build in a short time frame with mostly open source tools.

Specifically, there were no systems in place to build reliable, uniform, and reproducible pipelines for creating and managing training and prediction data at scale. Prior to Michelangelo, it was not possible to train models larger than what would fit on data scientists’ desktop machines, and there was neither a standard place to store the results of training experiments nor an easy way to compare one experiment to another. Most importantly, there was no established path to deploying a model into production–in most cases, the relevant engineering team had to create a custom serving container specific to the project at hand. At the same time, we were starting to see signs of many of the ML anti-patterns documented by Scully et al.

Michelangelo is designed to address these gaps by standardizing the workflows and tools across teams though an end-to-end system that enables users across the company to easily build and operate machine learning systems at scale. Our goal was not only to solve these immediate problems, but also create a system that would grow with the business.

When we began building Michelangelo in mid 2015, we started by addressing the challenges around scalable model training and deployment to production serving containers. Then, we focused on building better systems for managing and sharing feature pipelines. More recently, the focus shifted to developer productivity–how to speed up the path from idea to first production model and the fast iterations that follow.

In the next section, we look at an example application to understand how Michelangelo has been used to build and deploy models to solve specific problems at Uber. While we highlight a specific use case for UberEATS, the platform manages dozens of similar models across the company for a variety of prediction use cases.

Use case: UberEATS estimated time of delivery model

UberEATS has several models running on Michelangelo, covering meal delivery time predictions, search rankings, search autocomplete, and restaurant rankings. The delivery time models predict how much time a meal will take to prepare and deliver before the order is issued and then again at each stage of the delivery process.

Predicting meal estimated time of delivery (ETD) is not simple. When an UberEATS customer places an order it is sent to the restaurant for processing. The restaurant then needs to acknowledge the order and prepare the meal which will take time depending on the complexity of the order and how busy the restaurant is. When the meal is close to being ready, an Uber delivery-partner is dispatched to pick up the meal. Then, the delivery-partner needs to get to the restaurant, find parking, walk inside to get the food, then walk back to the car, drive to the customer’s location (which depends on route, traffic, and other factors), find parking, and walk to the customer’s door to complete the delivery. The goal is to predict the total duration of this complex multi-stage process, as well as recalculate these time-to-delivery predictions at every step of the process.

On the Michelangelo platform, the UberEATS data scientists use gradient boosted decision tree regression models to predict this end-to-end delivery time. Features for the model include information from the request (e.g., time of day, delivery location), historical features (e.g. average meal prep time for the last seven days), and near-realtime calculated features (e.g., average meal prep time for the last one hour). Models are deployed across Uber’s data centers to Michelangelo model serving containers and are invoked via network requests by the UberEATS microservices. These predictions are displayed to UberEATS customers prior to ordering from a restaurant and as their meal is being prepared and delivered.

System architecture

Michelangelo consists of a mix of open source systems and components built in-house. The primary open sourced components used are HDFS, Spark, Samza, Cassandra, MLLib, XGBoost, and TensorFlow. We generally prefer to use mature open source options where possible, and will fork, customize, and contribute back as needed, though we sometimes build systems ourselves when open source solutions are not ideal for our use case.

Michelangelo is built on top of Uber’s data and compute infrastructure, providing a data lake that stores all of Uber’s transactional and logged data, Kafka brokers that aggregate logged messages from all Uber’s services, a Samza streaming compute engine, managed Cassandra clusters, and Uber’s in-house service provisioning and deployment tools.

In the next section, we walk through the layers of the system using the UberEATS ETD models as a case study to illustrate the technical details of Michelangelo.

Machine learning workflow

The same general workflow exists across almost all machine learning use cases at Uber regardless of the challenge at hand, including classification and regression, as well as time series forecasting. The workflow is generally implementation-agnostic, so easily expanded to support new algorithm types and frameworks, such as newer deep learning frameworks. It also applies across different deployment modes such as both online and offline (and in-car and in-phone) prediction use cases.

We designed Michelangelo specifically to provide scalable, reliable, reproducible, easy-to-use, and automated tools to address the following six-step workflow:  

1. Manage data
2. Train models
3. Evaluate models
4. Deploy models
5. Make predictions
6. Monitor predictions

Next, we go into detail about how Michelangelo’s architecture facilitates each stage of this workflow.

Manage data

Finding good features is often the hardest part of machine learning and we have found that building and managing data pipelines is typically one of the most costly pieces of a complete machine learning solution.

A platform should provide standard tools for building data pipelines to generate feature and label data sets for training (and re-training) and feature-only data sets for predicting. These tools should have deep integration with the company’s data lake or warehouses and with the company’s online data serving systems. The pipelines need to be scalable and performant,  incorporate integrated monitoring for data flow and data quality, and support both online and offline training and predicting. Ideally, they should also generate the features in a way that is shareable across teams to reduce duplicate work and increase data quality. They should also provide strong guard rails and controls to encourage and empower users to adopt best practices (e.g., making it easy to guarantee that the same data generation/preparation process is used at both training time and prediction time).

The data management components of Michelangelo are divided between online and offline pipelines. Currently, the offline pipelines are used to feed batch model training and batch prediction jobs and the online pipelines feed online, low latency predictions (and in the near future, online learning systems).

In addition, we added a layer of data management, a feature store that allows teams to share, discover, and use a highly curated set of features for their machine learning problems.  We found that many modeling problems at Uber use identical or similar features, and there is substantial value in enabling teams to share features between their own projects and for teams in different organizations to share features with each other.

Offline

Uber’s transactional and log data flows into an HDFS data lake and is easily accessible via Spark and Hive SQL compute jobs. We provide containers and scheduling to run regular jobs to compute features which can be made private to a project or published to the Feature Store (see below) and shared across teams, while batch jobs run on a schedule or a trigger and are integrated with data quality monitoring tools to quickly detect regressions in the pipeline–either due to local or upstream code or data issues.

Online

Models that are deployed online cannot access data stored in HDFS, and it is often difficult to compute some features in a performant manner directly from the online databases that back Uber’s production services (for instance, it is not possible to directly query the UberEATS order service to compute the average meal prep time for a restaurant over a specific period of time). Instead, we allow features needed for online models to be precomputed and stored in Cassandra where they can be read at low latency at prediction time.

We support two options for computing these online-served features, batch precompute and near-real-time compute, outlined below:

• Batch precompute. The first option for computing is to conduct bulk precomputing and loading historical features from HDFS into Cassandra on a regular basis. This is simple and efficient, and generally works well for historical features where it is acceptable for the features to only be updated every few hours or once a day. This system guarantees that the same data and batch pipeline is used for both training and serving. UberEATS uses this system for features like a ‘restaurant’s average meal preparation time over the last seven days.’
• Near-real-time compute. The second option is to publish relevant metrics to Kafka and then run Samza-based streaming compute jobs to generate aggregate features at low latency. These features are then written directly to Cassandra for serving and logged back to HDFS for future training jobs. Like the batch system, near-real-time compute ensures that the same data is used for training and serving. To avoid a cold start, we provide a tool to “backfill” this data and generate training data by running a batch job against historical logs. UberEATS uses this near-realtime pipeline for features like a ‘restaurant’s average meal preparation time over the last one hour.’

Shared feature store

We found great value in building a centralized Feature Store in which teams around Uber can create and manage canonical features to be used by their teams and shared with others. At a high level, it accomplishes two things:  

1. It allows users to easily add features they have built into a shared feature store, requiring only a small amount of extra metadata (owner, description, SLA, etc.) on top of what would be required for a feature generated for private, project-specific usage.
2. Once features are in the Feature Store, they are very easy to consume, both online and offline, by referencing a feature’s simple canonical name in the model configuration. Equipped with this information, the system handles joining in the correct HDFS data sets for model training or batch prediction and fetching the right value from Cassandra for online predictions.

At the moment, we have approximately 10,000 features in Feature Store that are used to accelerate machine learning projects, and teams across the company are adding new ones all the time. Features in the Feature Store are automatically calculated and updated daily.

In the future, we intend to explore the possibility of building an automated system to search through Feature Store and identify the most useful and important features for solving a given prediction problem.

Domain specific language for feature selection and transformation

Often the features generated by data pipelines or sent from a client service are not in the proper format for the model, and they may be missing values that need to be filled. Moreover, the model may only need a subset of features provided. In some cases, it may be more useful for the model to transform a timestamp into an hour-of-day or day-of-week to better capture seasonal patterns. In other cases, feature values may need to be normalized (e.g., subtract the mean and divide by standard deviation).

To address these issues, we created a DSL (domain specific language) that modelers use to select, transform, and combine the features that are sent to the model at training and prediction times. The DSL is implemented as sub-set of Scala. It is a pure functional language with a complete set of commonly used functions. With this DSL, we also provide the ability for customer teams to add their own user-defined functions. There are accessor functions that fetch feature values from the current context (data pipeline in the case of an offline model or current request from client in the case of an online model) or from the Feature Store.

It is important to note that the DSL expressions are part of the model configuration and the same expressions are applied at training time and at prediction time to help guarantee that the same final set of features is generated and sent to the model in both cases.

Train models

We currently support offline, large-scale distributed training of decision trees, linear and logistic models, unsupervised models (k-means), time series models, and deep neural networks. We regularly add new algorithms in response to customer need and as they are developed by Uber’s AI Labs and other internal researchers. In addition, we let customer teams add their own model types by providing custom training, evaluation, and serving code. The distributed model training system scales up to handle billions of samples and down to small datasets for quick iterations.

A model configuration specifies the model type, hyper-parameters, data source reference, and feature DSL expressions, as well as compute resource requirements (the number of machines, how much memory, whether or not to use GPUs, etc.). It is used to configure the training job, which is run on a YARN or Mesos cluster.

After the model is trained, performance metrics (e.g., ROC curve and PR curve) are computed and combined into a model evaluation report. At the end of training, the original configuration, the learned parameters, and the evaluation report are saved back to our model repository for analysis and deployment.

In addition to training single models, Michelangelo supports hyper-parameter search for all model types as well as partitioned models. With partitioned models, we automatically partition the training data based on configuration from the user and then train one model per partition, falling back to a parent model when needed (e.g. training one model per city and falling back to a country-level model when an accurate city-level model cannot be achieved).

Training jobs can be configured and managed through a web UI or an API, often via Jupyter notebook. Many teams use the API and workflow tools to schedule regular re-training of their models.

Evaluate models

Models are often trained as part of a methodical exploration process to identify the set of features, algorithms, and hyper-parameters that create the best model for their problem. Before arriving at the ideal model for a given use case, it is not uncommon to train hundreds of models that do not make the cut. Though not ultimately used in production, the performance of these models guide engineers towards the model configuration that results in the best model performance. Keeping track of these trained models (e.g. who trained them and when, on what data set, with which hyper-parameters, etc.), evaluating them, and comparing them to each other are typically big challenges when dealing with so many models and present opportunities for the platform to add a lot of value.

For every model that is trained in Michelangelo, we store a versioned object in our model repository in Cassandra that contains a record of:

• Who trained the model
• Start and end time of the training job
• Full model configuration (features used, hyper-parameter values, etc.)
• Reference to training and test data sets
• Distribution and relative importance of each feature
• Model accuracy metrics
• Standard charts and graphs for each model type (e.g. ROC curve, PR curve, and confusion matrix for a binary classifier)
• Full learned parameters of the model
• Summary statistics for model visualization

The information is easily available to the user through a web UI and programmatically through an API, both for inspecting the details of an individual model and for comparing one or more models with each other.

Model accuracy report

The model accuracy report for a regression model shows standard accuracy metrics and charts. Classification models would display a different set, as depicted below in Figures 4 and 5:

Decision tree visualization

For important model types, we provide sophisticated visualization tools to help modelers understand why a model behaves as it does, as well as to help debug it if necessary. In the case of decision tree models, we let the user browse through each of the individual trees to see their relative importance to the overall model, their split points, the importance of each feature to a particular tree, and the distribution of data at each split, among other variables. The user can specify feature values and the visualization will depict the triggered paths down the decision trees, the prediction per tree, and the overall prediction for the model, as pictured in Figure 6 below: 

Feature report

Michelangelo provides a feature report that shows each feature in order of importance to the model along with partial dependence plots and distribution histograms. Selecting two features lets the user understand the feature interactions as a two-way partial dependence diagram, as showcased below:

Deploy models

Michelangelo has end-to-end support for managing model deployment via the UI or API and three modes in which a model can be deployed:

1. Offline deployment. The model is deployed to an offline container and run in a Spark job to generate batch predictions either on demand or on a repeating schedule.
2. Online deployment. The model is deployed to an online prediction service cluster (generally containing hundreds of machines behind a load balancer) where clients can send individual or batched prediction requests as network RPC calls.
3. Library deployment. We intend to launch a model that is deployed to a serving container that is embedded as a library in another service and invoked via a Java API. (It is not shown in Figure 8, below, but works similarly to online deployment).

In all cases, the required model artifacts (metadata files, model parameter files, and compiled DSL expressions) are packaged in a ZIP archive and copied to the relevant hosts across Uber’s data centers using our standard code deployment infrastructure. The prediction containers automatically load the new models from disk and start handling prediction requests.  

Many teams have automation scripts to schedule regular model retraining and deployment via Michelangelo’s API. In the case of the UberEATS delivery time models, training and deployment are triggered manually by data scientists and engineers through the web UI.

Make predictions

Once models are deployed and loaded by the serving container, they are used to make predictions based on feature data loaded from a data pipeline or directly from a client service. The raw features are passed through the compiled DSL expressions which can modify the raw features and/or fetch additional features from the Feature Store. The final feature vector is constructed and passed to the model for scoring. In the case of online models, the prediction is returned to the client service over the network. In the case of offline models, the predictions are written back to Hive where they can be consumed by downstream batch jobs or accessed by users directly through SQL-based query tools, as depicted below: 

Referencing models

More than one model can be deployed at the same time to a given serving container. This allows safe transitions from old models to new models and side-by-side A/B testing of models. At serving time, a model is identified by its UUID and an optional tag (or alias) that is specified during deployment. In the case of an online model, the client service sends the feature vector along with the model UUID or model tag that it wants to use; in the case of a tag, the container will generate the prediction using the model most recently deployed to that tag. In the case of batch models, all deployed models are used to score each batch data set and the prediction records contain the model UUID and optional tag so that consumers can filter as appropriate.

If both models have the same signature (i.e. expect the same set of features) when deploying a new model to replace an old model, users can deploy the new model to the same tag as the old model and the container will start using the new model immediately. This allows customers to update their models without requiring a change in their client code. Users can also deploy the new model using just its UUID and then modify a configuration in the client or intermediate service to gradually switch traffic from the old model UUID to the new one.

For A/B testing of models, users can simply deploy competing models either via UUIDs or tags and then use Uber’s experimentation framework from within the client service to send portions of the traffic to each model and track performance metrics.

Scale and latency

Since machine learning models are stateless and share nothing, they are trivial to scale out, both in online and offline serving modes. In the case of online models, we can simply add more hosts to the prediction service cluster and let the load balancer spread the load. In the case of offline predictions, we can add more Spark executors and let Spark manage the parallelism.

Online serving latency depends on model type and complexity and whether or not the model requires features from the Cassandra feature store. In the case of a model that does not need features from Cassandra, we typically see P95 latency of less than 5 milliseconds (ms). In the case of models that do require features from Cassandra, we typically see P95 latency of less than 10ms. The highest traffic models right now are serving more than 250,000 predictions per second.

Monitor predictions

When a model is trained and evaluated, historical data is always used. To make sure that a model is working well into the future, it is critical to monitor its predictions so as to ensure that the data pipelines are continuing to send accurate data and that production environment has not changed such that the model is no longer accurate.

To address this, Michelangelo can automatically log and optionally hold back a percentage of the predictions that it makes and then later join those predictions to the observed outcomes (or labels) generated by the data pipeline. With this information, we can generate ongoing, live measurements of model accuracy. In the case of a regression model, we publish R-squared/coefficient of determination, root mean square logarithmic error (RMSLE), root mean square error (RMSE), and mean absolute error metrics to Uber’s time series monitoring systems so that users can analyze charts over time and set threshold alerts, as depicted below:

Management plane, API, and web UI

The last important piece of the system is an API tier. This is the brains of the system. It consists of a management application that serves the web UI and network API and integrations with Uber’s system monitoring and alerting infrastructure. This tier also houses the workflow system that is used to orchestrate the batch data pipelines, training jobs, batch prediction jobs, and the deployment of models both to batch and online containers.

Users of Michelangelo interact directly with these components through the web UI, the REST API, and the monitoring and alerting tools.

Building on the Michelangelo platform

In the coming months, we plan to continue scaling and hardening the existing system to support both the growth of our set of customer teams and Uber’s business overall. As the platform layers mature, we plan to invest in higher level tools and services to drive democratization of machine learning and better support the needs of our business:

• AutoML. This will be a system for automatically searching and discovering model configurations (algorithm, feature sets, hyper-parameter values, etc.) that result in the best performing models for given modeling problems. The system would also automatically build the production data pipelines to generate the features and labels needed to power the models. We have addressed big pieces of this already with our Feature Store, our unified offline and online data pipelines, and hyper-parameter search feature. We plan to accelerate our earlier data science work through AutoML. The system would allow data scientists to specify a set of labels and an objective function, and then would make the most privacy-and security-aware use of Uber’s data to find the best model for the problem. The goal is to amplify data scientist productivity with smart tools that make their job easier.
• Model visualization.Understanding and debuggingmodels is increasingly important, especially for deep learning. While we have made some important first steps with visualization tools for tree-based models, much more needs to be done to enable data scientists to understand, debug, and tune their models and for users to trust the results.
• Online learning. Most of Uber’s machine learning models directly affect the Uber product in real time. This means they operate in the complex and ever-changing environment of moving things in the physical world. To keep our models accurate as this environment changes, our models need to change with it. Today, teams are regularly retraining their models in Michelangelo. A full platform solution to this use case involves easily updateable model types, faster training and evaluation architecture and pipelines, automated model validation and deployment, and sophisticated monitoring and alerting systems. Though a big project, early results suggest substantial potential gains from doing online learning right.
• Distributed deep learning. An increasing number of Uber’s machine learning systems are implementing deep learning technologies. The user workflow of defining and iterating on deep learning models is sufficiently different from the standard workflow such that it needs unique platform support. Deep learning use cases typically handle a larger quantity of data, and different hardware requirements (i.e. GPUs) motivate further investments into distributed learning and a tighter integration with a flexible resource management stack.

If you are interesting in tackling machine learning challenges that push the limits of scale, consider applying for a role on our team!  

Jeremy Hermann is an Engineering Manager and Mike Del Balso is a Product Manager on Uber’s Machine Learning Platform team.

Cheating is a given.

Inspectors certify that gas-station pumps are pumping unadulter­ated fuel and accurately reporting the count, and they put tamper-evident seals on the pumps that will alert them to attempts by station owners to fiddle the pumps in their favor. Same for voting machines, cash registers, and the scales at your grocery store.

The basic theory of cheating is to assume that the cheater is ‘‘rational’’ and won’t spend more to cheat than they could make from the scam: the cost of cheating is the risk of getting caught, multiplied by the cost of the punishment (fines, reputational dam­age), added to the technical expense associated with breaking the anti-cheat mechanisms.

Software changes the theory. Software – whose basic underlying mechanism is ‘‘If this happens, then do this, otherwise do that’’ – allows cheaters to be a lot more subtle, and thus harder to catch. Software can say, ‘‘If there’s a chance I’m undergoing inspection, then be totally honest – but cheat the rest of the time.’’

This presents profound challenges to our current regulatory model: Vegas slot machines could detect their location and if they believe that they are any­where near the Nevada Gaming Commission’s testing labs, run an honest payout. The rest of the time, they could get up to all sorts of penny-shaving shenanigans that add up to millions at scale for the casino owners or the slot-machine vendors (or both).

Even when these systems don’t overtly cheat, software lets them tilt the balance away from humans and towards corporations. The Nevada Gaming Commission sets the payout schedule for slot machines, but it doesn’t regulate the losses. This allows slot machine vendors to tune their machines so that a losing spin is much more likely to look like a ‘‘near miss’’ (lemon and two cherries, paying zero; three cherries pays a jackpot). The machine looks like it’s doing the same thing with a win or a loss, but losses are actually fine-tuned performances of near-win designed to confound your intuition about how close victory might be.

Software makes for a much more dangerous set of cheats, though. It’s one thing to be cheated by a merchant’s equipment: there are only so many merchants, and to operate a business, they have to submit themselves to spot inspections and undercover audits by secret shoppers.

But what happens when the things you own start to cheat you? The most famous version of this is Volkswagen’s Dieselgate scandal, which has cost the company billions (and counting): Volkswagen engineered several models of its diesel vehicles to detect when the engine was undergoing emissions testing and to tilt the engines’ performance in favor of low emis­sions (which also meant more fuel consumption). The rest of the time, the engines defaulted to a much more polluting mode that also yielded better gas mileage. Thus the cars were able to be certified as low-emissions by regulators and as high efficiency by reviewers and owners – having their cake and eating it too.

Dieselgate killed people, but the cheating in the Dieselgate scandal was still aimed at government inspectors. The next level of cheating comes when systems try to fool independent researchers.

A celebrated recent example of this came with the Wannacry ransomware epidemic. Wannacry is an old piece of malicious software, and it uses a variety of vectors to find and infect vulnerable hosts; once it takes root, Wannacry encrypts all its victims’ files, and demands a Bitcoin ransom in exchange for the decryption key. In early summer 2017, Wannacry had a resurgence after it was harnessed to a leaked NSA cyberweapon that made it much more virulent.

But within days of that resurgence, Wannacry was stopped dead in its tracks, thanks to the discovery and deployment of a ‘‘killswitch’’ built into the software. When Wannacry took over a new computer, the first thing it did is check to see whether it could get an answer when it tried to open a web-session to . If there was a web-server at that address, Wannacry ceased all operations. By registering this domain and standing up a web-server that answered to it, a security researcher was able to turn off Wannacry, everywhere in the world, all at once.

A casual observer may be puzzled by this kill switch. Why would a crimi­nal put such a thing in their software? The answer is: to cheat.

The greatest risk to a program like Wannacry is that a security researcher will be able to trick it into infecting a computer under the researcher’s control, a ‘‘honey pot’’ system that is actually a virtual machine – a computer program pretending to be a computer. Virtual machines are under their owners’ perfect control: everything the malicious software does within them can be inspected. Researchers use virtual machines like cyberpunk villains use VR: to trap their prey in a virtual world that is subjec­tively indistinguishable from objective reality, an Inception-style ruse that puts the malware under the researcher’s omnipotent microscope.

These head-in-a-jar virtual machines are often configured to pretend to be the entire internet as well. When the malware caught within them tries to reach a distant web-server, the researcher answers on that server’s behalf, to see if they can trick the malware into attempting to communicate with its master and so reveal its secrets.

Wannacry’s author tried to give their software the ability to distinguish a honey-pot from the real world. If the software’s attempt to contact the nonexistent domain was successful, then the software knew that it was trapped in a re­searcher’s lab where all queries were duly answered in an attempt to draw it out. If Wannacry got an answer from , it folded into a protective, encrypted foetal position and refused to uncurl. Registering the domain and standing up a web-server there tricked every new Wannacry infection in the world into thinking that it was running on a honey-pot system, so they all stopped working.

Wannacry was a precursor to a new kind of cheating: cheating the in­dependent investigator, rather than the government. Imagine that the next Dieselgate doesn’t attempt to trick the almighty pollution regulator (who has the power to visit billions in fines upon the cheater): instead, it tries to trick the reviewers, attempting to determine if it’s landed on a Car and Driver test-lot, and then switching into a high-pollution, high-fuel-efficiency mode. The rest of the time, it switches back to its default state: polluting less, burning more diesel.

This is already happening. MSI and Asus – two prominent vendors of computer graphics cards – have been repeatedly caught shipping hardware to reviewers whose software had been sped way, way up (‘‘overclocked’’) over the safe operating speed. These cards will run blazingly fast for the duration of the review process and a little while longer, before burning out and being rendered useless – but that will be long after the reviewers return them to the manufacturer. The reviewers advise their readers that these are much faster than competing cards, and readers shell out top dollar and wonder why they can’t match the performance they’ve read about in the reviews.

The cheating can be closer to home than that.

You’ve probably heard stories of inkjet cartridges that under-report their fill-levels, demanding that you throw them away and replace them while there’s still plenty of (precious and overpriced) ink inside of them. But that’s just for starters. In 2015, HP pushed a fake security update to millions of Officejet owners, which showed up as a routine, ‘‘You must update your soft­ware’’ notification on their printers’ screens. Running that update installed a new, secret feature in your printer, with a long fuse. After six months’ wait, the infected printers all checked to see whether their ink cartridges had been refilled, or manufactured by third parties, and to refuse to print with any ink that HP hadn’t given its corporate blessing to.

HP is an egregious cheater, and this kind of cheating is in the DNA of any company that makes its living selling consumables or service at extremely high markups – they do their business at war with their customers. The better the deal their customers get, the worse the deal is for the manufacturer, and so these products treat customers as enemies, untrusted parties who must be tricked or coerced into installing new versions of the manufacturer’s software (like the iTunes and Kindle ‘‘updates’’ that have removed features the products were sold with) and using only the manufacturer’s chosen consumables.

The mobile phone industry has long been at war with its customers. When phones were controlled primarily by carriers, they were designed to prevent customers from changing networks without buying a new phone, raising the cost on taking your busi­ness elsewhere. Apple wrested control back to itself, producing a phone that was locked primarily to its app store, so that the only way to sell software to an iPhone user was to give up 30% of the lifetime revenue that customer generated through the app. Carriers adapted custom versions of Android to lock customers to their networks with shovelware apps that couldn’t be removed from the home-screen and app store lock-in that forced customers to buy apps through their phone company.

What began with printers and spread to phones is coming to everything: this kind of technology has proliferated to smart thermostats (no apps that let you turn your AC cooler when the power company dials it up a couple degrees), tractors (no buying your parts from third-party companies), cars (no taking your GM to an independent mechanic), and many categories besides.

All these forms of cheating treat the owner of the device as an enemy of the company that made or sold it, to be thwarted, tricked, or forced into con­ducting their affairs in the best interest of the com­pany’s shareholders. To do this, they run programs and processes that attempt to hide themselves and their nature from their owners, and proxies for their owners (like reviewers and researchers).

Increasingly, cheating devices behave differ­ently depending on who is looking at them. When they believe themselves to be under close scrutiny, their behavior reverts to a more respectable, less egregious standard.

This is a shocking and ghastly turn of affairs, one that takes us back to the dark ages. Before the Englightenment, before the scientific method and its peer review, science was done by alchemists, who worked in secret.

Alchemists – like all humans – are mediocre lab-technicians. Without peer reviewers around to point out the flaws in their experiments, alchemists compounded their human frailty with bad experi­mental design. As a result, an alchemist might find that the same experiment would produce a ‘‘differ­ent outcome’’ every time.

In reality, the experiments lacked sufficient con­trols. But again, in the absence of a peer reviewer, alchemists were doomed to think up their own explanations for this mysterious variability in the natural world, and doomed again to have the self-serving logic of hubris infect these explanations.

That’s how alchemists came to believe that the world was haunted, that God, or the Devil, didn’t want them to understand the world. That the world actually rearranged itself when they weren’t looking to hide its workings from them. Angels punished them for trying to fly to the Sun. Devils tricked them when they tried to know the glory of God – indeed, Marcelo Rinesi from The Institute for Ethics and Emerging Technologies called modern computer science ‘‘applied demonology.’’

In the 21st century, we have come full circle. Non-human life forms – limited liability corpo­rations – are infecting the underpinnings of our ‘‘smart’’ homes and cities with devices that obey a different physics depending on who is using them and what they believe to be true about their surroundings.

What’s worse, 20th century law puts its thumb on the scales for these 21st century demons. The Computer Fraud and Abuse Act (1986) makes it a crime, with jail-time, to violate a company’s terms of service. Logging into a website under a fake ID to see if it behaves differently depending on who it is talking to is thus a potential felony, provided that doing so is banned in the small-print clickthrough agreement when you sign up.

Then there’s section 1201 of the Digital Millen­nium Copyright Act (1998), which makes it a felony to bypass the software controls access to a copy­righted work. Since all software is copyrightable, and since every smart gadget contains software, this allows manufacturers to threaten jail-terms for anyone who modifies their tractors to accept third-party carburetors (just add a software-based check to ensure that the part came from John Deere and not a rival), or changes their phone to accept an independent app store, or downloads some code to let them choose generic insulin for their implanted insulin pump.

The software in gadgets makes it very tempting indeed to fill them with pernicious demons, but these laws criminalize trying to exorcise those demons.

There’s some movement on this. A suit brought by the ACLU attempts to carve some legal exemp­tions for researchers out of the Computer Fraud and Abuse Act. Another suit brought by the Electronic Frontier Foundation seeks to invalidate Section 1201 of the Digital Millennium Copyright Act.

Getting rid of these laws is the first step towards restoring the order in which things you own treat you as their master, but it’s just the start. There must be anti-trust enforcement with the death penalty – corporate dissolution – for companies that are caught cheating. When the risk of getting caught is low, then increasing penalties are the best hedge against bad action. The alternative is toasters that won’t accept third-party bread and dishwashers that won’t wash unauthorized dishes.

Making better computers won’t solve the world’s problems, but none of the world’s problems are ours to solve for so long as the computers we rely on are sneaking around behind our backs, treating us as their enemies.

Radar images of asteroid 3122 Florence obtained at the 70-metre antenna at NASA’s Goldstone Deep Space Communications Complex between August 29 and September 1 have revealed that the asteroid has two small moons, and also confirmed that main asteroid Florence is about 4.5 km (2.8 miles) in size. Florence is only the third triple asteroid known in the near-Earth population out of more than 16,400 that have been discovered to date. All three near-Earth asteroid triples have been discovered with radar observations and Florence is the first seen since two moons were discovered around asteroid 1994 CC in June 2009.

The sizes of the two moons are not yet well known, but they are probably between 100 – 300 metres (300-1000 feet) across. The times required for each moon to revolve around Florence are also not yet known precisely but appear to be roughly 8 hours for the inner moon and 22 to 27 hours for the outer moon. The inner moon of the Florence system has the shortest orbital period of any of the moons of the 60 near-Earth asteroids known to have moons. In the Goldstone radar images, which have a resolution of 75 metres, the moons are only a few pixels in extent and do not reveal any detail.

The radar images also provide our first close-up view of Florence itself. Although the asteroid is fairly round, it has a ridge along its equator, at least one large crater, two large flat regions, and numerous other small-scale topographic features. The images also confirm that Florence rotates once every 2.4 hours, a result that was determined previously from optical measurements of the asteroid’s brightness variations.

The animated sequence to the left is built from a series of radar images of Florence. The sequence lasts several hours and shows more than two full rotations of the large, primary body. The moons can be clearly seen as they orbit the main body. Radar images are different from pictures taken with a digital camera but are similar to ultrasound images. The geometry in radar images is analogous to seeing an object from above its north pole with the illumination coming from the top. Projection effects can make the positions of Florence and its moons appear to overlap even though they are not touching.

Florence reached its closest approach to Earth early on September 1 and is now slowly receding from our planet. Additional radar observations are scheduled at NASA’s Goldstone Solar System Radar in California and at the National Science Foundation’s Arecibo Observatory in Puerto Rico through September 8. These observations should show more surface detail on Florence and provide more precise estimates of the orbital periods of the two moons. Those results are valuable to scientists because they can be used to estimate the total mass and density of the asteroid.

An animated view of asteroid Florence’s two moons is posted below.

Where once the term burnout was applied exclusively to health care workers, police officers, firefighters, paramedics or social workers who deal with trauma and human services — think Graham Greene’s novel “A Burnt-Out Case,” about a doctor in the Belgian Congo, a book that gave rise to the term colloquially — the term has since expanded to workers who are now part of a more connected, hyperactive and overcompensating work force.

But occupational burnout goes beyond needing a simple vacation or a family retreat, and many experts, psychologists and institutions, including the Centers for Disease Control and Prevention, highlight long-term and unresolvable burnout as not a symptom but rather a major health concern. (Though it does not appear in the Diagnostic Statistic Manual, the benchmark for psychological ailments, it does appear in the International Statistical Classification of Diseases and Related Health Problems, a classification used by the World Health Organization.)

“We’re shooting ourselves in the foot,” Ms. Seppala told me. “Biologically we are not meant to be in that high-stress mode all the time. We got lost in this idea that the only way to be productive is to be on the go-go-go mode.”

It is difficult to identify burnout, which often feels like surrender or failure rather than what it really is: a chronic disease.

Common Work Stressors

• Overcoming challenges associated with new software, changing atmospheres or different processes

• Unrealistic deadlines

• Frequent scheduling conflicts or interruptions

• Unpredictable schedules

• Physical demands like exposure to weather or heavy lifting

• Added responsibility beyond the initial scope of one’s role while not being compensated for the supervision

• Interpersonal demands such as interactions with colleagues or customers

These stressors can manifest in outbursts against co-workers, violence or anger toward loved ones at home, loss of appetite and passion for things once loved, or being unable to find motivation for things that you were able to accomplish with ease.

The pioneering researcher behind the study of burnout since the 1970s, Christina Maslach, professor emerita of psychology at the University of California, Berkeley, conducted a study that concluded there are three major signs of workplace burnout.

Signs of Burnout to Watch For

According to Dr. Maslach’s study, common signs of burnout include:

• Feeling emotionally drained and mentally unwell. Nausea. Being unable to sleep or constantly fighting sicknesses like head colds.

• Feeling alienated by your colleagues and bosses, feeling constantly underappreciated, or feeling ostracized by them.

• Feeling you are not personally achieving your best, or regularly “phoning it in.”

“There are a lot of things that can happen when people begin to have this problem at work,” Dr. Maslach said. “There are things like absenteeism, turnover, but also things in terms of errors, not being careful about the work they’re doing. We see a lot of difficulty with people getting along with each other — angry, aggressive.”

In 1981, Dr. Maslach and her colleagues devised the “Maslach Burnout Inventory.” It is still used today to determine whether an employee is experiencing burnout that may be causing performance or personal issues.

“Stress phenomenon can take a toll in terms of physical health, which can then get tied into absenteeism. There are a number of ways in which the quality of the work goes down as a result of burnout,” she said.

If anyone complains about it, she said, they’re automatically labeled a wimp.

“Quite honestly in America we glorify stress,” Dr. Maslach said. “And that’s another thing that leads people to be quiet and shut up about some of the stressors they’re facing because they don’t want to be viewed about not doing their best.”

Ways to Combat Burnout at Work

If you’re suffering from burnout at work, or if any of those symptoms sound familiar to you, there are a few things you can do now, before you get some time off to recover. (Although you should definitely consider some time off to recover, if you can.)

• Focused breathing, which can tap into your parasympathetic nervous system to help you reduce or manage stress.

• Frequent breaks, preferably five-minute breaks for every 20 minutes spent on a single task, or sitting at your desk.

• Ergonomic chairs and desks, like a sit-stand arrangement, or even a small plant in your office space.

• A trusted mentor at work with whom you can discuss and strategize other ways to deal with work-related issues.

• A hobby outside of work through which you can decompress, de-stress and dissociate from work. It doesn’t have to be anything specific, but regular exercise or another fitness activity works wonders here, and has benefits beyond stress relief.

If you have the ability to work remotely, that’s another great way to add stress-reducers to your life. Periodically working out of the office enables you to try working from a quiet and contemplative space in which creativity may grow. It could also allow for more time outdoors. But broadly speaking, Dr. Maslach said, it is the human connection that’s most effective at combating burnout.

“What we found is that people’s health, well-being, everything in life, is way better if you’re connected with other people,” she said. “That social network, that each of you have each other’s back, that they’re there for you and you’re there for them, that’s like money in the bank. That’s a precious, precious resource.”

Jason Lang is the team leader of Workplace Health Programs within the C.D.C. He says that aside from good diet, exercise and sleep, there’s one surefire way to combat general malaise, job dissatisfaction, low morale and burnout.

“Laughter,” he said. “Find some humor in daily life.”

Continue reading the main story

(Submitted on 12 Apr 2017)

Abstract: Legacy encryption systems depend on sharing a key (public or private) among the peers involved in exchanging an encrypted message. However, this approach poses privacy concerns. The users or service providers with the key have exclusive rights on the data. Especially with popular cloud services, the control over the privacy of the sensitive data is lost. Even when the keys are not shared, the encrypted material is shared with a third party that does not necessarily need to access the content. Indeed, Homomorphic Encryption (HE), a special kind of encryption scheme, can address these concerns as it allows any third party to operate on the encrypted data without decrypting it in advance. Although this extremely useful feature of the HE scheme has been known for over 30 years, the first plausible and achievableFully Homomorphic Encryption (FHE) scheme, which allows any computable function to perform on the encrypted data, was introduced by Craig Gentry in 2009. Even though this was a major achievement, different implementations so far demonstrated that FHE still needs to be improved significantly to be practical on every platform. Therefore, this survey focuses on HE and FHE schemes. First, we present the basics of HE and the details of the well-known Partially Homomorphic Encryption (PHE) and Somewhat Homomorphic Encryption (SWHE), which are important pillars of achieving FHE. Then, the main FHE families, which have become the base for the other follow-up FHE schemes are presented.Furthermore, the implementations and new improvements in Gentry-type FHE schemes are also surveyed. Finally, further research directions are discussed. We believe this survey can give a clear knowledge and foundation to researchers and practitioners interested in knowing, applying, as well as extending the state of the art HE, PHE, SWHE, and FHE systems.

grizzly

Writing scalable server applications in the Java™ programming language has always been difficult. Before the advent of the Java New I/O API (NIO), thread management issues made it impossible for a …

Java 79 14

You: “So what do I do with this free quantum computer you gave me access to???”

Us: “More now than ever before!”

We just rolled out a huge collection (20+) of brand new Jupyter notebooks on our QISKit (Quantum Information Software developer Kit). Whether you’re a quantum research scientist, trying to study the fundamentals of quantum states, a developer who wants to understand how you’d actually solve problems in chemistry or optimization, or someone who is curious how to build quantum games, you’re gonna love what we just put out. You are hereby invited to add to the already 1 million-plus experiments run on the system.

Quantum research scientists: we’ve updated QISKit with important new functionality for studying tomography and decoherence, a.k.a. experiments that let you study the state of the quantum system. We also built new tools to help you visualize quantum states. Plus, a brand new compiler to map the experiments you actually want to run onto the hardware we have.

Quantum developers, we’ve told you in the past we think chemistry and optimization will be some of the earliest use cases for quantum. Want to see how to actually run chemistry and optimization experiments? We just published a Jupyter notebook that shows you how to calculate the equilibrium bond lengths in Hydrogen (H₂), and Lithium hydride (LiH). Over time, we’ll expand the range of molecules and properties we can simulate.

How about optimization? In the real world, there are so many uses of optimization (think of all the things you want to minimize and maximize!). We just put out a Jupyter notebook that showcases MaxCut problems, defined by finding the minimum (cost of something), and the maximum (the profit of that something), as well as the Traveling Salesman challenge of the “perfect route.”

Quantum applications for the classical world

So how do we envision using these notebooks? Possibly as part of a larger project, that maybe you’re doing with the Data Science Experience (DSX, an interactive, collaborative, cloud-based environment for data scientists). Just in case that’s in the cards for you, we put out a notebook that helps you get started doing quantum experiments through the DSX environment. We also make it easy to invite fellow researchers to collaborate or simply share your work within the community.

Sometimes it is all fun and (quantum) games

These notebooks are just the tip of the iceberg of our recent upgrade to QISkit. But we’ve also had some great contributions by our friends and community members all over the world. Dr. James Wootton from the University of Basel, for example, posted a “how to” guide for writing quantum games, like Quantum Battleships. Here’s how he puts it on his blog: “Like normal Battleships, but simpler and more complex at the same time.”

The IBM Q experience takes you into the lab, into the depths of the dilution refrigerator, through the magic of the cloud, and is freely available to anyone curious about programming a quantum computer. To get started, visit our user guide. And then jump to our Jupyter notebooks to start experimenting.

The quest to restore the American elm has been underway for more than half a century. Today, with help from The Nature Conservancy’s Christian Marks, success is closer than ever—which is good news for our floodplain forests, as well as our urban communities.

On a humid day in mid-June, Jessica Colby is hunched over a collection of bright green stems, each one waving a leaf or two, each one a tiny banner of hope. It’s almost noon, and the temperatures in the greenhouses at the University of Massachusetts in Amherst are climbing. Colby pushes her hair off her damp brow and, wielding a small blade, gently scrapes the outer layer from the stem she is holding, before dipping it in rooting powder and adding it to the lineup of cuttings anchored before her in a moistened block of foam. The leafy stems march in straight rows across the table, like a band of miniature soldiers fighting for a cause.

Colby and her fellow interns, Izzy Bazluke and Lisette Stone, are, in fact, the latest recruits in what has been a long battle to bring back one of America’s most iconic trees, Ulmus americana. Their fearless leader, Christian Marks, a floodplain ecologist with The Nature Conservancy’s Connecticut River Program, has spent more than a decade tackling floodplain restoration in the Northeast, with a special focus on the American elm. Along the way, he’s developed an eye for what he calls survivor trees—like the two on Elwell Island in Northampton, Massachusetts, where Colby’s stems were harvested. “It’s sort of an obsession with me now,” he says. “I’m always watching, always looking.” As he drives New England’s winding roads, Marks peers into the forests, past the shadows, looking for the elm’s distinctive vase-shaped form, rising above the canopy. These massive trees—often 100 feet tall and more than 100 years old—stand alone in the forest now, surrounded by smaller dead and dying elms, reminders of another time.

Loss of an Icon

The tragedy that befell one of the country’s most magnificent trees began nearly a century ago. First described in the Netherlands in 1919, the Dutch elm fungus is carried by the European elm bark beetle, which crossed the Atlantic in 1930 in a shipment of logs purchased by an Ohio furniture maker. Before long, Dutch elm disease (Ophiostoma ulmi) was sweeping across much of the eastern United States. One after another, the giant trees were infected—and then swiftly cut down. Their famously arched silhouettes, which once soared in shady cathedrals of intertwining branches along countless community main streets, vanished, leaving behind barren stretches of pavement and neighborhoods stripped of beloved trees that had stood for generations. By the 1980s, millions of elms had been lost. The devastation continued along the region’s riverbanks, where the elm had been an anchor species in the floodplain forests, providing critical habitat for wildlife and protecting human communities from rising waters during severe storms.

Today, most of us have never stood beneath one of these iconic trees. Perhaps our grandparents remember them. Or maybe we’ve visited New York’s Central Park or the National Mall in Washington, DC, where fungicide treatments and generous endowments have helped to preserve a few remaining noble specimens. But for most of us, the giant elm is the stuff of legend, a lost wonder of the natural world. And while young elms persist along our riverbanks, almost none survive long enough to reach the canopy. The ancient tree has vanished from the forest, too.

If Marks has his way, though, the legend will live again. Elms along our riverbanks will help to save an entire ecosystem of expansive floodplain forests, and generations to come will experience once more the profound beauty of these soaring trees along our city streets. But restoring a legend, it turns out, is not for the faint of heart. The process is labor intensive and fraught with uncertainty, demanding years of strict adherence to careful scientific methods—and endless reserves of patience. Success, ultimately, requires a leap of faith, an understanding that today’s efforts are a gift to the future, to forests unseen.

Labor of Love

In the sweltering greenhouses, interns Colby and Bazluke methodically complete row after row of cuttings—about 270 by the end of the day. “We thought the whole propagation process would take about a week,” says Stone, who has overseen the painstaking effort. “But it took four.” In the end, the three interns, with help from a band of volunteers, completed 4,827 cuttings. Only 25 percent were expected to take root. But three weeks later, more than 90 percent had successfully rooted, defying expectations. “It was a great problem to have!” says Stone, who then worked nearly round the clock to get all the rooted cuttings potted. Those that survive, will eventually be transplanted into trial plots, where they will be tested for disease-resistance.

This is the secret to successful restoration, explains Marks. “We need new varieties—genetic diversity.” For more than 60 years, researchers have been scrambling to develop this diversity, searching for varieties that can withstand Dutch elm disease, including a second more virulent form of the pathogen (O. nova-ulmi). “We now have seven varieties that demonstrate good tolerance,” says Marks. Two of these, Princeton and Valley Forge, were successful enough that they are now sold in nurseries and are slowly being reintroduced into the urban landscape. “But seven varieties is not enough,” says Marks. “We need to double that number for long-term survival in the wild.”

Jim Slavicek is a research biologist at the USDA Forest Service’s lab in Delaware, Ohio, where he and his team have been working on elm restoration since 2003. “The DED fungus mutates,” says Slavicek, “which means it will eventually overcome the tolerance that exists in selections like Princeton and Valley Forge. That’s why we need to focus on the forested landscape, where the most DED-tolerant trees can naturally propagate and evolve, eventually developing mechanisms that can withstand future onslaughts of the fungus.” Urban settings, by contrast, typically feature single varieties, and the trees tend not to propagate themselves. Ultimately, restoration of naturally occurring forest elms will provide a source of disease-tolerant trees for use in urban settings, too.

No one knows the ecology of floodplain forests better than Marks, who has spent years pulling on waders and exploring the Connecticut River watershed, often tramping through the forest understory in waist-high water. At nearly every site he’s studied—more than 100 in all, elms are still taking root, but they are quickly succumbing to disease, never reaching the soaring heights of centuries past. “We realized that we have to figure out how to get this species back, if we are truly going to restore these floodplains,” says Kim Lutz, director of the Connecticut River Program. And so began a partnership. The Forest Service had the facilities, skills, and staffing to do the meticulous breeding needed to establish new varieties. The Conservancy had land where new trees could be planted. And they had Christian Marks, with his intimate knowledge of New England floodplains and the region’s great survivor trees.

“Rainbow. Podunk River. Hadley. Whale Tails. Cummingham. Goff Brook—I’ve collected from all of them,” says Marks, reeling off the names of some of the ancient elms he has stood beneath each spring, watching as arborists climb 60 or 80 feet up into the spreading limbs. The clipped branches spiral to the ground, where Marks waits to collect and label them. Of the 50 survivor elms Marks has worked with in New England, only about one in four will have elevated tolerance for the disease. His tireless mission, year after year, has been to figure out which ones might hold the promise of new hope for the American elm.

Some of the branches he collects provide the cuttings to make the clonal copies started by interns at work in the greenhouse. Others are rushed overnight to the US Forest Service research station in Ohio, for the start of the complex cross-breeding process. Here, they are recut and stored in water until the flowers open and drop their pollen onto the wax paper below. Next, USFS researchers collect the pollen and head out to the trial plots, where varieties of disease-resistant “mother” elms, including Princeton and Valley Forge, have been planted. Here, the fresh pollen from the ancient survivor “father” trees is carefully blown onto individual branches, each one encased and sealed in a plastic bag so as not to be contaminated by stray pollen from other trees. “Every father tree is crossed with at least two mothers,” says Marks. “We have more than 100 crosses at this point.”

A couple of months later, the USFS team collects the seeds produced by the latest cross and rushes them back to Marks in New England, where they are planted in tiny pots—and carefully labeled. Row after row of seedlings line the tables now at the UMass greenhouses, some no bigger than a fingernail, others nearing three feet in height—about 800 in all. With help from her team, Stone checks their progress every day, noting signs of growth invisible to the casual observer. For months, the whole batch of tiny seedlings, along with the thousands of cuttings, will be carefully tended and repotted, before they are ready for transplanting to test plots in the field. Then comes the waiting and watching.

For the next several years, at sites in Vermont, New Hampshire, Connecticut, and Massachusetts, Conservancy interns will monitor the progress of the fragile elms. This summer, Colby and Bazluke spend long days in the field hacking their way through thick vegetation to reach the saplings, battling tall grasses, giant ostrich ferns, and tangles of thistle, bindweed, and stinging nettle. They measure basal diameter and height. They look for tags, making sure each tree is labeled. They spray deer repellent and check to be sure every trunk is wrapped for protection from rodents. They slash away at any competing weeds. And they keep an eye out for yellow and wilting leaves—signs of DED.

“It’s pretty wild to see all these little trees planted out there,” says Gus Goodwin, conservation coordinator for The Nature Conservancy’s Vermont chapter. “And to realize how long it will be before we’ll have any definitive answers about survival rates. But it’s a great feeling to be part of this process.” Goodwin, who is also a woodworker with an appreciation for the unique characteristics of elm wood, adapted an existing app that has dramatically improved the quality and efficiency of the field work for Marks and his team. “Essentially, it allows elm techs to use a high-accuracy GPS to precisely map where the trees are planted,” he says, “which means less time spent searching for hidden saplings!” The app also makes it easy to enter all the growth, health, and vigor data out in the field, and then synch it with the database back at the lab for speedy editing—a quantum leap from the laborious task of re-entering data from hand-recorded sheets, notes Goodwin. “And then double- and triple-checking every entry for accuracy.”

When the tiny elms reach 1 inch in diameter, the injections begin—every single sapling gets a shot of the two strains of DED. “That is the moment of truth,” says Marks. “We can tell in a couple of weeks whether they will be disease resistant. Most are dead by the end of the summer.” Those that survive—perhaps a third—will continue to grow and put down roots, pioneers in establishing a new population of wild elms.

Urgent Mission

Restoring a lost species requires a certain kind of humility and tenacity. The work is laborious. The failure rate is high. The timeline is long. But Christian Marks is not easily daunted. He has a vision of what’s possible—and he loves these trees. “I’m just lending a helping hand early in the process,” he explains, describing what he likes to call his dating service for elm trees. “Eventually, we will walk away, and the evolution of resistance by natural selection will take over.”

But first, he explains, urgency in his voice, we need to establish enough disease-tolerant trees in the same place so that they can breed among one another. The recent outbreak of a new invasive pest, the emerald ash borer, which is attacking another critical floodplain species, makes the restoration and replanting of elms along rivers in the Northeast more urgent than ever. “The good news,” says Marks, “is that we have several new varieties in the pipeline that look promising. We are nearing the finish line.”

A recent grant from the Manton Foundation is helping to speed the restoration process along, including the flurry of propagating work in the greenhouse this summer. “Thanks to their generosity, we are in the midst of a five-year effort to see if we can double the number of disease-resistant cultivars,” says Lutz.

Progress can’t come soon enough. As the threat of climate change continues to loom, more and more people are beginning to understand why floodplains are so important, why we need these giant protective sponges along our riverbanks to buffer human communities from rising waters. Floodplain forests also filter sediment and excess nutrients to protect water quality, and they provide habitat for wildlife and recreation for people. Urban areas, too, are poised to benefit from the return of the elm, as cities seek green solutions to climate change. Studies have shown, for example, that a single large American elm, located on the southern side of a home, can intercept 2,384 gallons of storm water, conserve 107 kWh of energy, and sequester 518 pounds of CO2 annually.

“No matter where I go,” says Mark Smith, deputy director for The Nature Conservancy’s North America Water program, “the reality of increased floods and flooding damage is front and center.” The issue unites people, he notes, cutting across every geographical boundary and every party line. From the hills of California to the fields of Iowa to the coast of Louisiana to the banks of the Connecticut River in New England, communities everywhere are looking for solutions. “One of Christian’s great contributions to this effort,” says Smith, “has been his deep knowledge of floodplains and their dynamics and what’s needed to bring them back. And, of course, his commitment to the elm.”

Thanks to the help of dedicated interns and volunteers, Marks has planted elms at more than 30 sites in the Connecticut River watershed—3,300 seedlings from crosses at the Conservancy’s field trial plots in Vermont and more than 1,000 cuttings from clonal copies of the seven disease-resistant varieties. If some of this year’s 5,000 new cuttings and crosses are successful, more varieties will be joining the original seven. Enthusiasm among volunteers is high. “There’s something about planting a tree,” says Marks. “It’s meaningful to people. It creates a tangible legacy.”

Some of the restoration sites, like the Conservancy’s Maidstone Bends Preserve on the Upper Connecticut River in Vermont, have been successful, boasting elms that now stand more than 20 feet tall. Other sites have struggled. One was destroyed by fire. At another, dozens of young saplings snapped in two, when rising waters froze during a winter flood. But Marks perseveres.

Several new restoration projects are about to get underway, including two in New Hampshire—one along the Ashuelot River in Swanzey and another along the Connecticut River in Colebrook. In Massachusetts, after five years of collaboration with multiple partners, the Conservancy recently helped protect more than 370 acres of rare floodplain habitat just minutes from the city of Springfield. “The scope of this project is unprecedented,” says Marks, noting that, once the Conservancy’s restoration work on the parcel is complete—roughly 223 acres—the Fannie Stebbins Memorial Wildlife Refuge will be the largest expanse of natural floodplain in the Connecticut River watershed.

The restoration effort on the Stebbins land will feature the planting of thousands of trees, including disease-tolerant elms. With each sapling planted, hope will take root once again. Hope for a future where the American elm once again rises high above the forest canopy along our riverbanks. A future where children once again look up from the sidewalk and marvel at the graceful silhouette of a giant elm, shading the street below with its sheltering branches—its presence a quiet testament to the years of dedication and commitment that powered the quest to restore a beloved tree to the American landscape.

Suki Casanave is a writer for the philanthropy teams in Massachusetts and New Hampshire.

Image copyrightGetty Images

A harmful virus that can cause devastating brain damage in babies could offer up a surprising new treatment for adult brain cancer, according to US scientists.

Until now, Zika has been seen only as a global health threat - not a remedy.

But latest research shows the virus can selectively infect and kill hard-to-treat cancerous cells in adult brains.

Zika injections shrank aggressive tumours in fully grown mice, yet left other brain cells unscathed.

Human trials are still a way off, but experts believe Zika virus could potentially be injected into the brain at the same time as surgery to remove life-threatening tumours, the Journal of Experimental Medicine reports.

The Zika treatment appears to work on human cell samples in the lab.

Image copyrightGetty Images

There are many different types of brain cancer. Glioblastomas are the most common in adults and one of the trickiest to treat.

They are fast growing and diffuse, meaning they spread through the brain, making it difficult to see where the tumour ends and the healthy tissue begins.

Radiotherapy, chemotherapy and surgery may not be enough to remove these invasive cancers.

But the latest research, in living mice and donated human brain tissue samples, shows Zika therapy can kill cells that tend to be resistant to current treatments.

It is thought that these glioblastoma stem cells continue to grow and divide, producing new tumour cells even after aggressive medical treatment.

Different, healthy stem cells are found in abundance in baby brains, which probably explains why regular Zika can be so damaging to infants, say the researchers.

Adult brains, however, have very few stem cells. This means Zika treatment should destroy only the cancer-causing brain stem cells without causing much collateral damage.

As an extra safety precaution, the team, from Washington University School of Medicine and the University of California San Diego School of Medicine, have already begun modifying the virus to make it more tame than regular Zika.

Researcher Dr Michael Diamond said: "Once we add a few more changes, I think it's going to be impossible for the virus to overcome them and cause disease.

"It looks like there's a silver lining to Zika. This virus that targets cells that are very important for brain growth in babies, we could use that now to target growing tumours."

He hopes to begin human trials within 18 months.

Using viruses to fight cancer is not a new idea, but using Zika as the weapon of choice is.

UK scientists at the University of Cambridge are beginning similar trials with Zika.

Dr Catherine Pickworth, from Cancer Research UK, said: "This promising research shows that a modified version of the Zika virus can attack brain tumour cells in the lab.

"This could one day lead to new treatments for this particularly hard to treat type of cancer."

---

Image copyrightReuters

• Zika is a virus people can catch if they are bitten by an infected mosquito
• Most people will have few or no symptoms, but the disease can pose a serious threat to babies in the womb
• Affected infants have been born with abnormally small heads and underdeveloped brains - a condition known as microcephaly
• The infection has been linked to severe birth defects in almost 30 countries
• Although Zika is no longer "an international medical emergency", the World Health Organization says it is closely monitoring the infection

Download PDF

In this guide, I’ll walk through what Cassandra generally is — and is not — its specific capabilities, and some recent examples of Cassandra’s use in projects at Insight Data Engineering (where I was a Fellow during the summer of 2017) and in industry.

I hope this guide is helpful for future Insight Data Engineering Fellows as they refine their projects and build their pipelines (especially Fellows who don’t have a lot of experience with distributed computing), as well as members of the public interested in data engineering!

This is not meant to be a comprehensive explanation of pipeline development with Cassandra, and as always, the documentation is the best place to start for answering technical questions, including installation and configuration.

What Cassandra is — and is not

Cassandra is a distributed NoSQL data storage system from Apache that is highly scalable and designed to manage very large amounts of structured data. It provides high availability with no single point of failure.

Cassandra’s data model is a partitioned row store with tunable consistency where each row is an instance of a column family that follows the same schema, and the first component (a component is a Cassandra data type) of a table’s primary key is the partition key.

Cassandra has been designed from the ground up with scalability and write speed in mind, which is a design choice reflected in the partitioning structure of a Cassandra keyspace. A keyspace is analogous to a SQL database, and a column family is analogous to a relational database table. For the rest of this guide, I’ll use the terms “database” and “table” to refer to a Cassandra keyspace and column family.

When first learning about the storage and computational resources required by Cassandra, I found it helpful to throw out my idea of relational database tables — Cassandra tables are stored as nested sorted maps, with all the write speed and overhead requirements that data structure implies. Within a partition, rows are stored by the remaining columns of the key and may be indexed separately, a parameter tunable within the Cassandra configuration.

This partitioning paradigm is the key to Cassandra’s fast write speed and a key tradeoff to consider compared to a relational database — because no matter how simple a Cassandra table query is, read/write efficiency is entirely dependent on the amount of partitions that must be accessed to answer a query.

Another basic property of Cassandra tables is the replication factor, which is the total number of nodes a copy of a row exists on. There is no master node in a Cassandra cluster, so the replication factor is easily tunable and Cassandra can support a replication factor of 1 (minimum) or more for any data type.

The replication factor is a key consideration when setting up a Cassandra cluster, as different use cases will have different needs for consistency and fault tolerance. For a very basic illustration of cluster size, replication factor, write/read level, and their impacts on cluster performance, please see this calculator. Modern Cassandra clusters also use a virtual node system to improve load balance for high volumes of reads to the cluster and compensate for heterogeneous machines.

Gossip is the message system that Cassandra nodes/virtual nodes use to make their data consistent with each other, and is used to enforce the replication factor in a cluster. Imagine a Cassandra cluster as a ring system where each node contains certain partitions of each table in a database and can only communicate with adjacent nodes.

Because of the replication factor, reads/writes in Cassandra are often managed by quorum, which is the last important concept I’d like to outline in this section.

Quorum is a tunable parameter where a strict majority of the nodes in the cluster must agree on the existing value of an object in a table to accept a read or write request. With the read level, R, and write level, W, of a cluster configurable for a cluster of size, N (e.g.; N is 5 for a 5-node cluster), if R+W>N, the cluster is said to be strongly consistent — a desirable property for many use cases where the user can be confident that writes and reads they make to any node will propagate reliably to the entire cluster.

Whether a cluster has strong or weak consistency, it will always be eventually consistent. Weakly consistent systems depend on quorum to evaluate timestamps of values and resolve conflicts in the event of partial writes. The key tradeoff is that strongly consistent systems have higher overhead for a standard read/write operation in terms of time to consistency.

One more thing: Cassandra is not a silver bullet for all your data storage needs, and is not the best choice for any arbitrary distributed database project. If you have a strong requirement for immediate consistency accompanied by high read speed, there are other databases with robust ACID tolerance for highly structured data, and you could consider playing to their strengths. Or, if you are ingesting data from sources of different priority where quorum cannot be used to evaluate write priority, Cassandra is not for you!

Other NoSQL applications specialize in different areas, and traditional RDBMS are not out of style yet for a good reason.

Recent Insight projects using Cassandra

Insight Data Engineering Fellow Adam Costarino used Cassandra in his project, Reddit Rank, a tool to map user influence and construct a graph to analyze subreddit similarity. Cassandra was an important part of his pipeline because he needed to construct a large number of adjacency lists — large enough to contain the entire post history on Reddit from the past 7 years! In his tables, adj_year was used to construct adjacency lists for distinct users by year. Adam chose a final Cassandra cluster size of 5 with a replication factor of 3 to accommodate the large amount of reads queries (via Spark) that his Cassandra tables would need.

Another Fellow, Kyle Schmidt used Cassandra in his project, Instabrand, to evaluate user relationships for a large Instagram-like social network. He fabricated user data from the Instagram API and used Cassandra to store information on events (likes, follows, and comments) and make results accessible to a web-based interface. His Cassandra cluster was able to manage real-time stream processing and writes to Cassandra from Spark for millions of users.

Yours truly also used Cassandra in my Insight project, discorecs. My project used release data from Discogs (a vinyl record database and marketplace) along with synthesized activity for 10M users to power a recommendation engine for new friends and album purchases. Cassandra was used to store the results from my recommendation calculations (in Spark) as well as a copy of the release metadata in Discogs to display on my web UI. Cassandra was run with full replication on 5 nodes in order to support writes from my batch and stream processing in Spark.

Recent enterprise projects using Cassandra in industry

Raptor Smart Advisor uses Cassandra to read and write user behavior for thousands of clients, along with Spark to batch process updates from data mining models. The linear scaling of nodes and easy retrieval of simple queries with CQLSH have allowed them to scale their service rapidly to create personalized recommendations for relevant products and improve UX on their clients’ sites.

AdVoice is a telecom-focused ad network and uses Cassandra to seamlessly serve ads to clients and monitor customers’ audio/text inventory in real time in an accessible environment. Along with Cassandra’s scalability and functionality, they use enterprise features only present in DataStax Enterprise to monitor analytics tools and enable in-memory caching of table views for clusters of variable size.

SmarterWeb has implemented an electronic invoicing system for a rapidly growing client base of Mexican and Central American customers in Cassandra. Easy integration with Solr and scalability of the cluster were key considerations for their use case, and Cassandra was able to compensate for issues they had with read consistency due to deadlock and timeouts in SQL Server and Azure.

Conclusion

Cassandra is a flexible and scalable tool that’s perfect for handling a high volume of incoming data. Because of its virtual node, replication factor, and partitioning paradigms, Cassandra is particularly well-suited to ingesting structured data and making it eventually consistent with a high degree of fault tolerance. While not perfect for every use case, it’s popular among Insight fellows and enterprise applications, and has many tutorials available online. I hope this guide has been a useful primer to Cassandra, and good luck to any future Insight Data Engineering Fellows reading this on your projects! You’re gonna do great!

Special thanks to Adam and Kyle for the code snippets from their Insight projects, used here with permission.

Hi HN!

We are the co-founders of Lyrebird (https://lyrebird.ai/) and PhD students in AI at University of Montreal. We are building speech synthesis technologies to improve the way we communicate with computers. Right now, our key innovation is that we can copy the voice of someone else and make it say anything. The tech is still at its early stage but we believe that it is eventually going to make possible a wide range of new applications such as:

- reading loud text messages with the voice of the sender,

- reading audiobooks with the voice of your choice,

- giving a personalized digital voice to people who lost their voice due to a disease,

- allowing video game makers to have more customized dialogs generated on the fly, or avatars of their players,

- allowing movie makers to freeze the voice of their actors so that they can still use it if the actor ages or dies.

Yesterday we launched a beta version of our voice-cloning software: anyone can record one minute of audio and get a digital voice that sounds like them.

We know that many on HN are concerned about potential misuses surrounding these technologies and we share your concern. We write further on our ethical stance on this page: https://lyrebird.ai/ethics/.

Our blogpost about the launch: https://lyrebird.ai/blog/create-your-voice-avatar that features the first video combining generated audio and generated elements of the video.

There was a thread about us on HN when we launched our website four months ago (https://news.ycombinator.com/item?id=14182262) but at that time, no one could test our software yet and we did not really answer any question of the community. So this time we are ready for questions and would love some feedback!