Thursday, August 28, 2014

ML EVENTS

GTC Express Webinar Program

http://www.gputechconf.com/resources/gtc-express-webinar-program

GTC ON DEMAND
RECORDINGS AND SLIDES

http://on-demand-gtc.gputechconf.com/gtcnew/on-demand-gtc.php

Machine Learning and AI

Yann LeCun
http://on-demand-gtc.gputechconf.com/gtcnew/on-demand-gtc.php?searchByKeyword=yann&searchItems=&sessionTopic=&sessionEvent=&sessionYear=2014&sessionFormat=&submit=&select=+

Clarifai: Enabling Next Generation Intelligent Applications
Matthew Zeiler (Clarifai)
Significant advances have recently been made in the fields of machine learning and image recognition, impacted greatly by the use of NVIDIA GPUs. Leading performance is harnessed from deep neural networks trained on millions of images to predict thousands of categories of objects. Our expertise at Clarifai in deep neural networks helped us achieve the world's best published image labeling results [ImageNet 2013]. We use NVIDIA GPUs to train large neural networks within practical time constraints and are creating a developer API to enable the next generation of applications in a variety of fields. This talk will describe what these neural networks learn from natural images and how they can be applied to auto-tagging new images, searching large untagged photo collections, and detecting near-duplicates. A live demo of our state of the art system will showcase these capabilities and allow audience interaction.

- See more at: http://on-demand-gtc.gputechconf.com/gtcnew/on-demand-gtc.php?searchByKeyword=clarifai&searchItems=&sessionTopic=&sessionEvent=&sessionYear=&sessionFormat=&submit=&select=+#sthash.e2roqQJ6.dpufhttp://on-demand.gputechconf.com/gtc/2014/video/S4959-clarifai-enabling-generation-intelligent-applications.mp4

VLAB at Stanford 

October 16, 2014

Datasciense from sky

Date/Time
Date(s) - Tuesday, September 16, 2014
6:00 pm - 8:30 pm


Deep Learning | VLAB Sept. 2014
A machine learning approach inspired by the human brain, Deep Learning is taking many industries by storm. Empowered by the latest generation of commodity computing, Deep Learning begins to derive significant value from Big Data. It has already radically improved the computer’s ability to recognize speech and identify objects in images, two fundamental hallmarks of human intelligence. 

Industry giants such as Google, Facebook, and Baidu have acquired most of the dominant players in this space to improve their product offerings. At the same time, startup entrepreneurs are creating a new paradigm, Intelligence as a Service, by providing APIs that democratize access to Deep Learning algorithms. 

Join us on September 16, 2014 to learn more about this exciting new technology and to be introduced to some of the new application domains, the business models, and the key players in this emerging field. 
https://www.vlab.org/events/deep-learning-unbounded-intelligence-as-a-service/



MLconf 2014 Parc 55 Wyndham, San Francisco. Friday, November 14th
MLconf SF Speakers

http://mlconf.com/mlconf-sf-speakers/


DataSummit
San Francisco, DogPatch Studios, October 30

https://import.io/data-summit

innovation summit calendar
http://theinnovationenterprise.com/media/W1siZiIsImdsb2JhbF9jb25maWdzL0FsbCBjb21wcmVzc2VkLnBkZiJdXQ/All%20compressed.pdf


NVIDIA TensorRT
High performance deep learning inference for production deployment
https://developer.nvidia.com/tensorrt



Posted by at No comments:

Tuesday, August 26, 2014

BLOGS


Susan J. Fowler
https://www.susanjfowler.com

Brainscape - flashcards for all exams
https://www.brainscape.com/

How to overcome a terror of public speaking
By Katie Hope Business reporter, BBC News
http://www.bbc.com/news/business-35397409?ocid=socialflow_twitter


ACT!
Brigitte_Gabrie founded the American Congress For Truth and ACT! for America, a citizen action network that promotes "national security and the defense of American democratic values against the assault of Radical Islam
http://www.actforamericaeducation.com/welcome/

http://en.wikipedia.org/wiki/Brigitte_Gabriel

Monash Research Blog
http://www.dbms2.com/2015/02/28/databricks-and-spark-update/


http://www.toptal.com/machine-learning/an-introduction-to-deep-learning-from-perceptrons-to-deep-networks#

Author of "Scala for Machine Learning" Packt publishing - October 2014
Data Science 101
http://patricknicolas.blogspot.com/
Learning To Be A Data Scientist
http://datascience101.wordpress.com/
http://www.searsmerritt.com/
Salience Entities
http://datasift.com/source/19/salience-entities
2016 : WHAT DO YOU CONSIDER THE MOST INTERESTING RECENT [SCIENTIFIC] NEWS? WHAT MAKES IT IMPORTANT?

Research affiliate, MIT Media Lab
Differentiable Programming

http://edge.org/response-detail/26794


Things to try after useR! – Part 1: Deep Learning with H2O

July 25, 2014

By Jo-fai Chow
http://www.r-bloggers.com/things-to-try-after-user-part-1-deep-learning-with-h2o/

TRAINING NEURAL NETWORKS ON THE GPU: INSTALLATION AND CONFIGURATION

http://rexdouglass.com/training-neural-networks-on-the-gpu-with-commodity-hardware-installation-and-configuration/

slide share


 H2O Distributed Deep Learning by Arno Candel 071614




what's new in java 8 programmer interview









java software design questions






public void reverseDLL( ) {
   Node temp=head; //swap head and tail
   head=tail; // head now points to tail
   tail=temp; //tail points to head
    //traverse the list swapping prev and next fields of each node
  Node p=head; //create a node and point to head

  while(p!=null) //while p does not equal null
    { //swap prev and next of current node
      temp=p.next; // p.next does that not equal null? confusing.
      p.next=p.prev; //this line makes sense since you have to reverse the link
      p.prev=temp; //having trouble visualizing this.
      p=p.next;//advance current node which makes sense
    }
 }


Career Websites
http://careercup.com/
http://careercup.com/video


Technical Program Manager Websites


http://www.fossygirl.com/resume/


https://sites.google.com/site/paultbarham/program-manager-resume

























data structures



final exam info






































































week 6 twosum



Python


















C#








discussion of twosum












java



sort array!



answer=427


































algorithm design and analysis Tim Roughgarden course reviews

















Cryptography







book


A Computational Introduction to Number Theory and Algebra

A book introducing basic concepts from computational number theory and algebra, including all the necessary mathematical background.



Cryptography 1



final -




weeks








exercises #1-#6





















BOOKS



High School Mathematics Extensions/Discrete Probability

https://en.wikibooks.org/wiki/High_School_Mathematics_Extensions/Discrete_Probability



A Computational Introduction to Number Theory and Algebra

http://shoup.net/ntb/ntb-v2.pdf


















Archive for the ‘Neural Networks’ Category






Is there any open source project implementing deep-learning algorithms taking advantages of GPU's massive parallel computing power?












Deep Learning on Hadoop



Adam Gibson - DL4J

includes RNTN for sentiment analysis










Deep learning on Hadoop/Spark -NextML








Caffe Tutorial

Convolutional Neural Network for image recognition










Statistical Learning in R
Stanford Online Course



Video Lectures






Promo Video:

https://www.youtube.com/watch?v=St2-97n7atk

Course Page:

https://class.stanford.edu/courses/HumanitiesandScience/StatLearning/Winter2015/about



course text "An Introduction to Statistical Learning, with Applications in R" (James, Witten, Hastie, Tibshirani - Springer 2013). 

http://www-bcf.usc.edu/~gareth/ISL/



R download

http://cran.us.r-project.org/



RStudio 

http://www.rstudio.com/










GAME THEORY








Entrepreneurial resources








Hiring Blog








coding in paradise









50 Questions to Test True Data Science Knowledge










EXPLORE PRIVATELY AND INFORMALLY WITH TOP STARTUP FOUNDERS AND THEIR VCS

https://specializedtypes.com/hyperlink/explore/





Petition:







Hamas  leaders must be tried for War Crimes –

















































































Posted by



at





No comments:
  

































          

        

          

        

Saturday, August 16, 2014


          

        















INTENSIVE COURSES in data science and ML



Free 6 week intensive course



http://insightdatascience.com/

http://insightdatascience.com/Insight_White_Paper_2014.pdf



http://insightdataengineering.com/

http://insightdataengineering.com/Insight_Data_Engineering_White_Paper_2014.pdf



http://www.sfbayacm.org/event/data-mining-quick-start

8 am - 5 pm, September 27, 2014, eBay
















Posted by



at





No comments:
  















































CONFERENCES



GPU, Silicon Valley, March 17-20, 2015

http://www.gputechconf.com/





WEBINARS







For those of you who are fans of work by the trio, below are links to their upcoming webinar:



Deep Neural Networks for Visual Pattern Recognition
Dan Ciresan, Senior Researcher, IDSIA
August 6, 2014 at 9:00 AM Pacific



10 Billion Parameter Neural Networks in Your Basement
Adam Coates, Director, Baidu
August 20, 2014 at 9:00 AM Pacific



Convolutional Networks: Machine Learning for Computer Perception
Yann LeCun, Director, AI Research, Facebook & Silver Professor, NYU
September 24, 2014 at 9:00 AM Pacific










Posted by



at





No comments:
  

































          

        

          

        

Thursday, August 14, 2014


          

        









NLP Parsers




















1. Stanford

The Stanford NLP (Natural Language Processing) Group

Neural Network, Java

http://nlp.stanford.edu/software/lex-parser.shtml



Stanford Parser FAQ


http://nlp.stanford.edu/software/parser-faq.shtml

Stanford Deterministic Coreference Resolution System.



Probabilistic Context-Free Grammars (PCFGs)

http://www.cs.columbia.edu/~mcollins/courses/nlp2011/notes/pcfgs.pdf



Alphabetical list of part-of-speech tags used in the Penn Treebank Project:

https://www.ling.upenn.edu/courses/Fall_2003/ling001/penn_treebank_pos.html







2. Google SyntaxNet Parsey Mc Parsey

Feed forward Neural Network, Python




GitHub SyntaxNet: Neural Models of Syntax

https://github.com/tensorflow/models/tree/master/syntaxnet

A TensorFlow implementation of the models described in

Andor et al. (2016)

http://arxiv.org/pdf/1603.06042v1.pdf





http://9to5google.com/2016/05/12/google-open-sources-parsey-mcparseface-the-worlds-most-accurate-parser/



http://www.theverge.com/2016/5/12/11666414/google-parsey-mcparseface-tensorflow-open-source-language-tool



Google NLP Research

http://research.google.com/pubs/NaturalLanguageProcessing.html







3. Yoav Goldberg & Eliyahu Kiperwasser  

Bi-directional LSTM, Python

BIST Parsers

Graph & Transition based dependency parsers using BiLSTM feature extractors

The techniques behind the parser are described in the paper

Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations.

http://arxiv.org/pdf/1603.04351v1.pdf

Eliyahu Kiperwasser
Computer Science Department
Bar-Ilan University
Ramat-Gan, Israel
elikip@gmail.com
Yoav Goldberg
Computer Science Department
Bar-Ilan University
Ramat-Gan, Israel
yoav.goldberg@gmail.com



 Abstract
We present a simple and effective scheme
for dependency parsing which is based on
bidirectional-LSTMs (BiLSTMs). Each sentence
token is associated with a BiLSTM vector
representing the token in its sentential context,
and feature vectors are constructed by
concatenating a few BiLSTM vectors. The
BiLSTM is trained jointly with the parser objective,
resulting in very effective feature extractors
for parsing. We demonstrate the effectiveness
of the approach by applying it to
a greedy transition based parser as well as to
a globally optimized graph-based parser. The
resulting parsers have very simple architectures,
and match or surpass the state-of-the-art
accuracies on English and Chinese.



Our
proposed feature extractors are based on a bidirectional
recurrent neural network (BiRNN), an extension
of RNNs that take into account both the past
x1:i and the future xi:n. We use a specific flavor
of RNN called a long short-term memory network
(LSTM)



Going deeper A deep RNN (or k-layer RNN) is
composed of k RNN functions RNN1, · · · , RNNk
that feed into each other: the output h
ℓ
1:n
of RNNℓ
becomes the input of RNNℓ+1. Stacking RNNs in
this way was empirically shown to be effective. Finally,
in a deep bidirectional RNN, both RNNF and
RNNR are k-layer RNNs, and BIRNNℓ
(x1:n, i) =
v
ℓ
i = h
ℓ
F,i ◦ h
ℓ
R,i. In this work, we use BiRNNs and
deep-BiRNNs interchangeably, specifying the number
of layers when needed.



 Historical Notes RNNs were introduced by
Elamn (Elman, 1990), and extended to BiRNNs
by (Schuster and Paliwal, 1997). The LSTM variant
of RNNs is due to (Hochreiter and Schmidhuber,
1997). BiLSTMs were recently popularized by
Graves (2008), and deep BiRNNs were introduced
to NLP by Irsoy and Cardie (2014), who used them
for sequence tagging.



Required software

Python 2.7 interpreter

PyCNN library

https://github.com/elikip/bist-parser



see additional NLP refs at https://levyomer.wordpress.com/2016/05/01/annotating-relation-inference-in-context-via-question-answering/



independent outlets, blogs


Parsing English in 500 Lines of Python












Tutorials



NLP Programming Tutorial 8 -
Phrase Structure Parsing
Graham Neubig
Nara Institute of Science and Technology (NAIST)



http://www.phontron.com/slides/nlp-programming-en-10-parsing.pdf


Word Corpus can be created with Common Crawl
We build and maintain an open repository of web crawl data that can be accessed and analyzed by anyone.
http://commoncrawl.org/












Posted by



at





No comments:
  

































          

        

          

        

Monday, August 11, 2014


          

        










ML PROJECTS and FRAMEWORKS


















DataBricks



http://databricks.com/







MLBase










ML Optimizer: This layer aims to automating the task of ML pipeline construction. The optimizer solves a search problem over feature extractors and ML algorithms included in MLI and MLlib. The ML Optimizer is currently under active development.







MLI: An experimental API for feature extraction and algorithm development that introduces high-level ML programming abstractions. A prototype of MLI has been implemented against Spark, and serves as a testbed for MLlib.







MLlib: Apache Spark's distributed ML library. MLlib was initially developed as part of the MLbase project, and the library is currently supported by the Spark community. Many features in MLlib have been borrowed from ML Optimizer and MLI, e.g., the model and algorithm APIs, multimodel training, sparse data support, design of local / distributed matrices, etc.













MLI: An API for Distributed Machine Learning








MLI is an Application Programming Interface designed to address the challenges of building Machine Learning algorithms in a distributed setting based on data-centric computing. Its primary goal is to simplify the development of high-performance, scalable, distributed algorithms. Our initial results show that, relative to existing systems, this interface can be used to build distributed implementations of a wide variety of common Machine Learning algorithms with minimal complexity and highly competitive performance and scalability.



From the database community, projects like MADLib [12] and Hazy [13]

have tried to expose ML algorithms in the context of well

established systems. Alternatively, projects like Weka [14],

scikit-learn [15] and Google Predict [16] have sought to expose

a library of ML tools in an intuitive interface. However, none

of these systems focus on the challenges of scaling ML to the



emerging distributed data setting.











GitHub




status of mlbase/mli



Evan R. Sparks

Hi there,
MLlib is the first component of MLbase - MLI and the higher levels of the stack are still being developed. Look for updates in terms of our progress on the hyperparameter tuning/model selection problem in the next month or so!
- Evan Apr 01, 2014










Evan Sparks github https://github.com/etrain

http://etrain.github.io/about.html

SPARKS at cs dot berkeley dot edu.



Patrick Wendell https://github.com/pwendell



Machine Learning Library (MLlib)












BIDMach - an interactive, general machine learning toolkit for Big Data





http://bid2.berkeley.edu/bid-data-project/


http://www.meetup.com/Silicon-Valley-Machine-Learning/events/197169132/
Its a data-centered world now, and machine learning is the key to getting value from data.  But we believe much of the value from Big Data is untapped, and requires better tools that are much faster, more agile and more tunable (allowing tailoring of models). The current wave of tools rely primarily on cluster computing for scale-up. The BID Data project focuses on *single-node performance first* and fully taps the latest hardware developments in graphics processors. It turns out this approach is faster in absolute terms for most problems (i.e. our tool on a graphics processor outperforms all cluster implementations on up to several hundred nodes), is fully interactive and supports direct prototype-to-production migration (no recoding). Some problems (e.g. training large deep learning networks), still benefit from scale-up on a cluster. We have developed a new family of communication primitives for large-scale ML which are provably close-to-optimal for a broad range of problems, and e.g. they hold the current record for distributed pagerank. Our most recent work is on live tuning and tailoring of models during optimization, and we have developed a new approach to optImization: parameter-cooled Gibbs sampling to support this.

John Canny

http://en.wikipedia.org/wiki/John_Canny





deeplearning4j

http://deeplearning4j.org/




A curated list of awesome machine learning frameworks, libraries and software (by language). Inspired by awesome-php.
Other awesome lists can be found in the [awesome-awesomeness](https://github.com/bayandin/awesome-awesomeness) list.



https://raw.githubusercontent.com/josephmisiti/awesome-machine-learning/master/README.md



http://deeplearning4j.org/word2vec.html

http://deeplearning4j.org/deepautoencoder.html

http://deeplearning4j.org/recursiveneuraltensornetwork.html



Caffe



Caffe is a deep learning framework developed with cleanliness, readability, and speed in mind. It was created by Yangqing Jia during his PhD at UC Berkeley, and is in active development by the Berkeley Vision and Learning Center (BVLC) and by community contributors. Caffe is released under the BSD 2-Clause license.

http://caffe.berkeleyvision.org/



Scala
Akka

http://akka.io/



ADDITIONAL TOOLS

FLUME



http://flume.apache.org/











Machine Learning stacks



FACTORIE



http://factorie.cs.umass.edu/

https://github.com/factorie/factorie



 ScalaNLP



http://www.scalanlp.org/

https://github.com/scalanlp



Numerical Libraries



 ScalaNLP Breeze



https://github.com/scalanlp/breeze

https://code.google.com/p/scalalab/wiki/BreezeAsScalaLabToolbox



 Spire

https://github.com/non/spire

http://typelevel.org/







 Saddle




https://github.com/saddle/saddle





Data mining with WEKA, Part 1: Introduction and regression

http://www.ibm.com/developerworks/library/os-weka1/



Data mining with WEKA, Part 2: Classification and clustering

http://www.ibm.com/developerworks/library/os-weka2/



JAVA NUMERIC COMPUTING

JBLAS

http://mikiobraun.github.io/jblas/javadoc/org/jblas/package-summary.html








By popular demand NVIDIA built a new powerful programming library, NVIDIA® cuDNN.



NVIDIA® cuDNN is a GPU-accelerated library of primitives for deep neural networks. It emphasizes performance, ease-of-use, and low memory overhead. NVIDIA cuDNN is designed to be integrated into higher-level machine learning frameworks, such as UC Berkeley's popularCaffe software. The simple, drop-in design allows developers to focus on designing and implementing neural net models rather than tuning for performance, while still achieving the high performance modern parallel computing hardware affords.



cuDNN is free for anyone to use for any purpose: academic, research or commercial. Just sign up for a registered CUDA developer account. Once your account is activated, log in and visit the cuDNN page at developer.nvidia.com/cuDNN. The included User Guide will help you use the library.



For any additional questions or to provide feedback, please contact us at cuDNN@nvidia.com. 







http://code.google.com/p/word2vec/



Where to obtain the training data



The quality of the word vectors increases significantly with amount of the training data. For research purposes, you can consider using data sets that are available on-line:








H2O is the world’s fastest in-memory platform for machine learning and predictive analytics on big data.

http://0xdata.com/h2o/

http://databricks.com/blog/2014/06/30/sparkling-water-h20-spark.html







BMRM



(Bundle Methods for Regularized Risk Minimization)



version 2.1





19 February 2009





http://users.cecs.anu.edu.au/~chteo/BMRM.html





PRESTO



ML Inside Presto Distributed SQL Query Engine

http://www.meetup.com/sfmachinelearning/events/218160592/



Presto is an open source distributed SQL query engine used by Facebook, in our Hadoop warehouse. It's typically about 10x faster than Hive, and can be extended to a number of other use cases. One of these extensions adds SQL functions to create and make predictions with machine learning models. The aim of this is to significantly reduce the time it takes to prototype a model, by moving the construction and testing of the model to the database.





Shiny

by RStudio

A web application framework for R

Turn your analyses into interactive web applications

No HTML, CSS, or JavaScript knowledge required



http://shiny.rstudio.com/





Applied Deep Learning for Vision and Natural Language with Torch7




TO UPLOAD SLIDES



OCTOBER 8



THURSDAY

9:00am PDT /12:00pm EDT



TORCH7: APPLIED DEEP LEARNING FOR VISION AND NATURAL LANGUAGE



Presenter: Nicholas Léonard

Element Inc., Research Engineer



This webinar is targeted at machine learning enthusiasts and researchers and covers applying deep learning techniques on classifying images and building language models, including convolutional and recurrent neural networks. The session is driven in Torch: a scientific computing platform that has great toolboxes for deep learning and optimization among others, and fast CUDA backends with multi-GPU support.



Presenter:

Nicholas Léonard, Research Engineer, Element Inc.

Presenter Bio:

Nicholas graduated from the Royal Military College of Canada in 2008 with a bachelor's degree in Computer Science. Nicholas Retired from the Canadian Army Officer Corp in 2012 to complete a Master's degree in Deep Learning at the University of Montreal. He currently applies deep learning to biometric authentication using smart phones.





cuDNN

https://developer.nvidia.com/cudnn



Key Features



cuDNN provides high performance building blocks for deep neural network applications, including:


    

  • Forward and backward convolution routines, including cross-correlation, designed for convolutional neural nets
    • 

  • Arbitrary dimension ordering, striding, and sub-regions for 4d tensors means easy integration into any neural net implementation
    • 

  • Forward and backward paths for many common layer types such as pooling, ReLU, Sigmoid, softmax and Tanh
    • 

  • Tensor transformation functions
    • 

  • Context-based API allows for easy multithreading
    • 

  • Optimized for the latest NVIDIA GPU architectures
    • 

  • Supported on Windows, Linux and MacOS systems with Kepler, Maxwell or Tegra K1 GPUs.
    • 






Watch the GPU-Accelerated Deep Learning with cuDNN webinar to learn more about cuDNN.





The convolution routines in cuDNN provide best-in-class performance while using almost no extra memory. cuDNN features customizable data layouts, flexible dimension ordering, striding, and sub-regions for the 4D tensors used as inputs and outputs to all of its routines. This flexibility avoids transposition steps to or from other internal representations. cuDNN also offers a context-based API that allows for easy multithreading and optional interoperability with CUDA streams.



References







15 Deep Learning Libraries

























































Posted by



at





No comments:
  

































        

      









Subscribe to:
Posts (Atom)