Logistics

• Will seek possible support from Intel, Amazon, MS, Google, others?
• Formally plan to make this a student-run organization
• Logistics: Monday 6-7
• Format: Tutorial/discussion + lightning talks
• Goal is to get this self-sustaining by the end of academic year
• Organizational models for learning about software
  • Courses, bootcamps, meetups, and communities of practice
  • http://www.thehackerwithin.org/
  • https://science.mozilla.org/
  • http://software-carpentry.org/index.html
  • http://labcarpentry.org/

Topics

• At the terminal
  • (2016-09-19) Local and remote terminals
  • (2016-09-19) Windows: Cygwin, MinGW, and Linux in Windows 10
  • (2016-09-19) SSH keys and password-free operation
  • (2016-09-26) screen, tmux, and other terminal multiplexing options
  • (2016-09-26) Terminal sharing with tmate
• Version control
  • (2016-09-19) Git, Mercurial, Subversion, and company
  • (2016-09-19) Hosted services: GitHub, Bitbucket, Assembla, etc
  • (2016-09-19) Cornell-local options: GitHub Enterprise, Sourceforge
  • (2016-09-19) What to control? Issues with built artifacts and large data
  • (2016-09-19) Locks, pull requests, and collaboration with version control
• Language environments and tooling
  • “Just enough” introductions:
    • (2016-09-26) Bash
    • (2016-10-03) Python
    • MATLAB
    • (2016-10-24) Julia
    • (2016-11-07) R
    • Issues in high-performance scripting
  • (2016-10-03) Using Jupyter notebooks
  • Hosted notebooks (Azure, Everware, Binder, Wakari, SageMathCloud, etc).
• Documentation
  • Javadoc,
  • (2016-11-28) Doxygen and codedocs.xyz
  • (2016-11-28) Sphinx and readthedocs
  • (2016-11-21) Plain-text formats: RESt Markdown, LaTeX
  • Collaborative document editing frameworks: Overleaf, Authorea, ShareLaTeX
  • (2016-11-21) Pandoc
  • Literate programming
• Scientific computing in the cloud
  • Working with
    • (2016-11-14) AWS/EC2
    • (2016-10-31) GCP
    • Azure
    • Red Cloud (get CAC tutorial?)
    • (2016-12-05) Accelerators, Infiniband, and other specialized support
• Cornell resources for scientific computing
  • CAC, CIT, RDMSG, and other groups
  • (2016-09-26) The Totient cluster
  • Local courses and workshops
  • Student-run tutorials
• Workflow automation and documentation
  • Make, Rake, SnakeMake, PyDoIt, etc
  • Pipelines and notebooks
  • Reproducibility
  • Continuous integration environments (Bamboo, Jenkins, TravisCI, etc)
• Getting computing resources for scientific computing
  • Overview: Servers, clusters, clouds, supercomputers
  • Cornell-local resources
  • Supercomputing resources
  • Cloud resources
• Environment virtualization
  • Virtual machines
  • Docker and containers
  • Language level: Python virtualenv and conda
• Reproducibility issues
  • Reproducibility best practices
  • Special issues with reproducibility and high-performance computing
• Build and configuration
  • “The build problem”: survey and history
  • Autotools, CMake, SCons, etc
• Packaging and distribution
  • Semantic versioning
  • Automating distribution
  • Packaging of compiled codes
  • Packaging in Python, Julia, R
  • Software licensing and copyright issues
• Data management
  • “Just enough” SQL
  • HDF5, NetCDF, and related formats
  • XML, YAML, JSON, metadata, and semi-structured data
  • Facilities for large-scale working data
  • Data set archival and dissemination: figshare, zenodo, Dryad, re3data, eCommons
• Plotting and visualization
  • Matplotlib, Seaborn, GNUPlot, D3, etc
  • Visualization of network data
  • Simulation visualization with VisIt and Paraview
  • XTK
• Testing and company
  • Linting code
  • Code reviews and tools
  • Valgrind and company
  • Types of tests: unit, integration, regression, etc
  • Test-driven development
  • Tooling for test automation
  • Continuous integration tools and services
  • Tickets and bug databases
  • Special issues in testing of numerical codes
• Monitoring and checkpointing
  • Assertions and exceptions
  • Logging systems
  • Automated checkpoints
  • Application-level checkpointing
• Tuning
  • “Just enough” computer architecture
  • Profiling for scripts and compiled codes
  • Profile-guided optimization
  • Sources for tuned libraries
  • Accelerator-aided libraries
  • Software modernization resources
• Mixed language programming
  • Models: script-driven, embedded scripting, configuration languages, cross-language library calls, code generation
  • Interface generators
  • Function calls vs inter-process communication
  • Build-time issues: portability, standard libraries, linkage
• Fast math frameworks and libraries
  • PETSc and Trilinos
  • ACTS collection