Abstract: Sentiment analysis is a common task in natural language processing that aims to
detect polarity of a text document (typically a consumer review). In the simplest settings, we
discriminate only between positive and negative sentiment, turning the task into a ...
T Mikolov, A Joulin, S Chopra, M Mathieu, MA Ranzato - arXiv preprint arXiv: …, 2014
Abstract: Recurrent neural network is a powerful model that learns temporal patterns in
sequential data. For a long time, it was believed that recurrent networks are difficult to train
using simple optimizers, such as stochastic gradient descent, due to the so-called ...
RICHARD SOCHER
SENTIMENT ANALYSIS online demo system at
http://nlp.stanford.edu:8080/sentiment/rntnDemo.html
CS224d: Deep Learning for Natural Language Processing
http://cs224d.stanford.edu/
NB Shah, A Parekh, S
Balakrishnan, K Ramchandran… - 2015
Page 1. Estimation
from Pairwise Comparisons: Sharp Minimax Bounds with Topology
Dependence Nihar B. Shah Abhay Parekh Sivaraman Balakrishnan Kannan
Ramchandran Joseph Bradley Martin Wainwright UC Berkeley Abstract ...
M Muraoka, S Shimaoka,
K Yamamoto, Y Watanabe… - 2014
Page 1. PACLIC 28 !65
Finding The Best Model Among Representative Compositional
Models Masayasu Muraoka† Sonse Shimaoka‡ Kazeto Yamamoto† Yotaro Watanabe†
Naoaki Okazaki†∗ Kentaro Inui† Tohoku
University ...
M Fan, Q Zhou, E
Chang, TF Zheng - 2014
Page 1. PACLIC 28 !328
Transition-based Knowledge Graph Embedding with Relational
Mapping Properties Miao Fan†,⇤,
Qiang Zhou†, Emily Chang‡, Thomas Fang Zheng†,⇧
†CSLT, Tsinghua National Laboratory for Information ...
Y Mroueh, E Marcheret,
V Goel - arXiv preprint arXiv:1501.05396, 2015
... In Issues in Visual and Audio-Visual Speech Processing.
MIT Press, 2004. [SCMN]
Richard Socher, Danqi Chen, Christopher D. Manning, and Andrew
Ng. Reasoning
with neural tensor networks for knowledge base completion. ...
GA Sigurdsson, S Hu
... IEEE, 2013. [5] Jia Deng, Wei Dong, Richard
Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet:
A large-scale hierarchical image database. ... Unsupervised discovery of
mid-level discrim- inative
patches. Computer VisionECCV 2012, 2012. [16] Richard Socher and
Li Fei-Fei. ...
A Sorgente, VC
Flegrei, G Vettigli, F Mele
... USA. ACM. Richard Socher, Alex
Perelygin, Jean Wu, Jason Chuang, Christopher
D. Manning, Andrew Y. Ng, and Christopher Potts. 2013. Recursive deep mod-
els for semantic compositionality over a sentiment treebank. ...
DTN NGUYEN, Y KIYOKI -
Information Modelling and Knowledge Bases XXVI, 2014
Page 336. Information
Modelling and Knowledge Bases XXVI B. Thalheim et al.(Eds.)
324 IOS Press, 2014 © 2014 The authors and IOS Press. All rights reserved. doi:
10.3233/978-1-61499-472-5-324 An Adaptive Search Path ...
A Abdelsadek - 2014
Page 1. DISTRIBUTED
INDEX FOR MATCHING MULTIMEDIA OBJECTS by Ahmed Abdelsadek
B.Sc., Cairo University, 2010 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE
REQUIREMENTS FOR THE DEGREE OF Master of Science in the ...
W Tang, Z Shi, Y Wu - 2015
Page 1. Research Report: A Unified Framework for Salient Object Detection of
Single/Multiple Images Based on Object Distributions at Semantic Level Wei Tang,
Zhenwei Shi, Ying Wu January 19, 2015 1 Motivation Conventionally ...
A Lazaridou, NT Pham, M Baroni - arXiv preprint arXiv:1501.02598, 2015
Page 1. Combining Language and Vision with a Multimodal Skip-gram Model
AngelikiLazaridou NghiaThePham MarcoBaroni Center for Mind/Brain Sciences University
of Trento {angeliki.lazaridou|thenghia.pham|marco.baroni}@unitn.it Abstract ...
DH Phan, TD Cao - Proceedings of the Fifth Symposium on Information and …, 2014
... NIPS 2013: 3111-3119. [5] Richard Socher and Christopher Manning. Tutorials on Deep
Learning for NLP. In procedding of NAACL HLT, 2013. ... St. Catherine's College, 2005.
[13] Richard Socher, Christopher D. Manning, Andrew Y. Ng. ...
A Lazaridou, G Dinu, A Liska, M Baroni - arXiv preprint arXiv:1501.02714, 2015
Page 1. From visual attributes to adjectives through decompositional distributional semantics
AngelikiLazaridou GeorgianaDinu AdamLiska MarcoBaroni Center for Mind/Brain Sciences
University of Trento {angeliki.lazaridou|georgiana.dinu|adam.liska|marco.baroni}@unitn.it ...
AC Gyllensten, M Sahlgren - arXiv preprint arXiv:1501.02670, 2015
Page 1. Navigating the Semantic Horizon using Relative Neighborhood Graphs
Amaru Cuba Gyllensten and Magnus Sahlgren Gavagai Bondegatan 21 116 33
Stockholm Sweden {amaru|mange}@gavagai.se Abstract This ...
J Lee - 2015
... [PL08] B. Pang and L. Lee. 2008. Opinion mining and sentiment analysis. Foundations and
Trends in Information Retrieval, 2(1-2):1– 135. [SHP*11] Richard Socher, Eric H Huang, Jeffrey
Pennington, Andrew Y Ng, and Christo- pher D Manning. 2011a. ...
J Woo, K Kitani, S Kim, H Kwak, W Shim - IS&T/SPIE Electronic Imaging, 2015
... We plan to solve these problems in the future work. REFERENCES [1] Li-Jia Li, Richard
Socher, and Li Fei-Fei, “Towards Total Scene Understanding: Classification, Annotation
and Segmentation in an Automatic Framework,” Proc. ...
R Eshleman, H Yang - Big Data and Cloud Computing (BdCloud), 2014 IEEE …, 2014
Page 1. Abstract— Twitter data has been applied to address a wide range of
applications (eg, political election prediction and disease tracking); however, no
studies have been conducted to explore the interactions and potential ...
P Vossen, T Caselli, F Ilievski, R Izquierdo, A Lopopolo…
... In Pro- ceedings of the 9th International Conference on Se- mantic Systems, I-SEMANTICS '13,
pages 121– 124, New York, NY, USA. ACM. Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai
Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hi- erarchical image database. ...
S Arora, Y Li, Y Liang, T Ma, A Risteski - arXiv preprint arXiv:1502.03520, 2015
Page 1. Random Walks on Context Spaces: Towards an Explanation of the Mysteries of
Semantic Word Embeddings Sanjeev Arora ∗ Yuanzhi Li † Yingyu Liang ‡ Tengyu Ma § Andrej
Risteski ¶ February 13, 2015 Abstract The papers of Mikolov et al. ...
M Gardner
Page 1. Thesis Proposal Combining Compositional and Latent Factorization Methods
for Knowledge Base Inference Matt Gardner Abstract A recent focus in natural
language processing research has been on the creation of ...
Yoshua Bengio
Scholar Alert: New articles in Yoshua Bengio's profile
K Xu, J Ba, R Kiros, A Courville, R Salakhutdinov… - arXiv preprint arXiv: …, 2015
Abstract: Inspired by recent work in machine translation and object detection, we introduce
an attention based model that automatically learns to describe the content of images. We
describe how we can train this model in a deterministic manner using standard ...
J Chung, C Gulcehre, K Cho, Y Bengio - arXiv preprint arXiv:1502.02367, 2015
Abstract: In this work, we propose a novel recurrent neural network (RNN) architecture. The
proposed RNN, gated-feedback RNN (GF-RNN), extends the existing approach of stacking
multiple recurrent layers by allowing and controlling signals flowing from upper recurrent ...
Y Bengio, DH Lee, J Bornschein, Z Lin - arXiv preprint arXiv:1502.04156, 2015
Abstract: Neuroscientists have long criticised deep learning algorithms as incompatible with
current knowledge of neurobiology. We explore more biologically plausible versions of deep
representation learning, focusing here mostly on unsupervised learning but developing a ...
YN Dauphin, H de Vries, J Chung, Y Bengio - arXiv preprint arXiv:1502.04390, 2015
Abstract: Parameter-specific adaptive learning rate methods are computationally efficient
ways to reduce the ill-conditioning problems encountered when training large deep
networks. Following recent work that strongly suggests that most of the critical points ...
Andrej Karpathy
BLOG
http://karpathy.github.io/
Nitish Srivastava, Elman Mansimov, Ruslan Salakhutdinov
http://www.cs.toronto.edu/~nitish/unsupervised_video/
Samuel Bowman
PhD Student, Stanford University
News
- I'm lecturing in CS 224U (Natural Language Understanding) and LING 1 this quarter. Expect slides soon.
- I proposed my dissertation under the tentative title of Realizing natural language semantics in learned representations.
- Interested in recursive NNs in MATLAB? I have a new release of my research code up, and I'm happy to offer support.
- Interested in training data for textual entailment? Be in touch, some colleagues and I collecting some!
- I'll be back at Google next summer to work with Oriol Vinyals on machine learning for language meaning.
- I just posted an expanded version of my manuscript on logical behavior in deep neural networks for language on arXiv.
- I have a new short paper up on arXiv on learning word vectors that encode lexical relationships that I recently presented at the AAAI Spring Symposium on Knowledge Representation and Reasoning.
Eduard Hovy
Carnegie Mellon University
www.springer.com/.../9783319080420...
Springer Science+Business Media
Applying the Methodology of Michael. Zock to Sentiment Analysis ... Sentiment analysis or opinion mining refers to the application of natural language
Language Production, Cognition,
and the Lexicon
inspired by Michael Zock (retired in 2013 after 30 years of research)
-
What are Sentiment, Affect, and Emotion? Applying the Methodology of Michael Zock to Sentiment Analysis
EH Hovy
Language Production, Cognition, and the Lexicon, 13-24
2015Retrofitting Word Vectors to Semantic Lexicons
M Faruqui, J Dodge, SK Jauhar, C Dyer, E Hovy, NA Smith
arXiv preprint arXiv:1411.4166
12014What a Nasty day: Exploring Mood-Weather Relationship from Twitter
J Li, X Wang, E Hovy
Proceedings of the 23rd ACM International Conference on Conference on ...
2014Sentiment Analysis on the People’s Daily
J Li, E Hovy
2014The C@ merata Task at MediaEval 2014: Natural language queries on classical music scores
R Sutcliffe, T Crawford, C Fox, DL Root, E Hovy
MediaEval 2014 Workshop, Barcelona, Spain
42014Application of Prize based on Sentence Length in Chunk-based Automatic Evaluation of Machine Translation
H Echizen’ya, K Araki, E Hovy
Proc. of the Ninth Workshop on Statistical Machine Translation, 381-386
12014Metaphor Detection through Term Relevance
M Schulder, E Hovy
ACL 2014, 18
2014Major Life Event Extraction from Twitter based on Congratulations/Condolences Speech Acts
J Li, A Ritter, C Cardie, E Hovy
Proceedings of Empirical Methods in Natural Language Processing
12014Data integration from open internet sources and network detection to combat underage sex trafficking
DR Silva, A Philpot, A Sundararajan, NM Bryan, E Hovy
Proceedings of the 15th Annual International Conference on Digital ...
2014A taxonomy and a knowledge portal for cybersecurity
D Klaper, E Hovy
Proceedings of the 15th Annual International Conference on Digital ...
2014Scoring coreference partitions of predicted mentions: A reference implementation
S Pradhan, X Luo, M Recasens, E Hovy, V Ng, M Strube
Proceedings of the Association for Computational Linguistics
32014286 The Functinal Perspective on Language and Discourse
MV Escandell Vidal, T Espigares, N Fabb, R Fawcett, Z Fenghui, A Fetzer, ...
The Functional Perspective on Language and Discourse: Applications and ...
2014Spatial compactness meets topical consistency: jointly modeling links and content for community detection
M Sachan, A Dubey, S Srivastava, EP Xing, E Hovy
Proceedings of the 7th ACM international conference on Web search and data ...
12014Automatic Post-Editing Method Using Translation Knowledge Based on Intuitive Common Parts Continuum for Statistical Machine Translation
H Echizen’ya, K Araki, Y Uchida, E Hovy
Speech and Computer, 129-136
2014Overview of CLEF QA Entrance Exams Task 2014
A Peñas, Y Miyao, Á Rodrigo, E Hovy, N Kando
CLEF
12014Recursive deep models for discourse parsing
J Li, R Li, E Hovy
Proceedings of the 2014 Conference on Empirical Methods in Natural Language ...
22014An extension of BLANC to system mentions
X Luo, S Pradhan, M Recasens, E Hovy
Proceedings of ACL, Baltimore, Maryland, June
12014Weakly Supervised User Profile Extraction from Twitter
J Li, A Ritter, E Hovy
ACL
62014RIPTIDE: Learning violation prediction models from boarding activity data
H Chalupsky, E Hovy
Technologies for Homeland Security (HST), 2013 IEEE International Conference ...
2013What Is a Paraphrase?
R Bhagat, E Hovy
Computational Linguistics 39 (3), 463-472
132013
Michael Zock
Research director at the CNRS, (LIF) university of Aix-Marseille
Verified email at lif.univ-mrs.fr
How Well Can a Corpus-Derived Co-Occurrence Network
Simulate Human Associative Behavior?
http://www.aclweb.org/anthology/W/W14/W14-0509.pdf
YOSHUA BENGIO'S ANSWER ON QUORA
I want to do an independent study on deep learning, but rather than some tutorial, I am interested in digging deep and implement the workings of a fundamental paper in this field for which code and data is available. Any direction will be deeply appreciated.
http://www.quora.com/What-are-some-fundamental-deep-learning-papers-for-which-code-and-data-is-available-to-reproduce-the-result-and-on-the-way-grasp-deep-learning/answer/Yoshua-Bengio?srid=dCMG&share=1
Here are some (paper, code url) pairs from deep learning research:...
Ben Sandbank
Refining Generative Language Models using Discriminative Learning
Ben Sandbank
Blavatnik School of Computer Science
Tel-Aviv University
Tel-Aviv 69978, Israel
sandban@post.tau.ac.il
Generative Language Model:
Language modeling is a fundamental task in natural
language processing and is routinely employed
in a wide range of applications, such as speech
recognition, machine translation, etc’. Traditionally,
a language model is a probabilistic model
which assigns a probability value to a sentence or a
sequence of words. We refer to these as generative
language models. A very popular example of a
generative language model is the n-gram, which
conditions the probability of the next word on the
previous (n-1)-words.
discriminative language model:
Although simple and widely-applicable, it has
proven difficult to allow n-grams, and other forms
of generative language models as well, to take advantage
of non-local and overlapping features.1
These sorts of features, however, pose no problem
for standard discriminative learning methods, e.g.
large-margin classifiers. For this reason, a new
class of language model, the discriminative language
model, has been proposed recently to augment
generative language models (Gao et al.,
2005; Roark et al., 2007). Instead of providing
probability values, discriminative language models
directly classify sentences as either correct or incorrect,
where the definition of correctness depends
on the application (e.g. grammatical /
ungrammatical, correct translation / incorrect translation,
etc').
Discriminative learning methods require
negative samples. Given that the corpora used for
training language models contain only real
sentences, i.e. positive samples, obtaining these
can be problematic.
Hal Daumé
natural language processing blog
Hyperparameter search, Bayesian optimization and related topics
ML Scalability to millions of features
Carnegie Mellon University
Petuum: A Framework for Iterative-Convergent Distributed ML
AMPLabs, Berkeley
Distributed Machine Learning and Graph Processing with Sparse Matrices
Paper #83
https://amplab.cs.berkeley.edu/wp-content/uploads/2013/03/eurosys13-paper83.pdf
Adam Gibson
SlideShare
http://www.slideshare.net/agibsonccc/ir-34811120
Chong Wang
Jointly modeling aspects, ratings and sentiments for movie recommendation (JMARS)
Q Diao, M Qiu, CY Wu, AJ Smola, J Jiang, C Wang
Proceedings of the 20th ACM SIGKDD international conference on Knowledge ...
| | 2014 |
Dynamic Language Models for Streaming Text
D Yogatama, C Wang, BR Routledge, NA Smith, EP Xing
Transactions of the Association for Computational Linguistics 2, 181-192
| | 2014 |
Personalized collaborative clustering
Y Yue, C Wang, K El-Arini, C Guestrin
Proceedings of the 23rd international conference on World wide web, 75-84
| 1 | 2014 |
Community Specific Temporal Topic Discovery from Social Media
Z Hu, C Wang, J Yao, E Xing, H Yin, B Cui
arXiv preprint arXiv:1312.0860
| | 2013 |
Asymptotically exact, embarrassingly parallel MCMC
W Neiswanger, C Wang, E Xing
arXiv preprint arXiv:1311.4780
| 29 | 2013 |
A Nested HDP for Hierarchical Topic Models
J Paisley, C Wang, D Blei, MI Jordan
arXiv preprint arXiv:1301.3570
| | 2013 |
Modeling overlapping communities with node popularities
PK Gopalan, C Wang, D Blei
Advances in Neural Information Processing Systems, 2850-2858
| 3 | 2013 |
Variance reduction for stochastic gradient optimization
C Wang, X Chen, AJ Smola, EP Xing
Advances in Neural Information Processing Systems, 181-189
| 11 | 2013 |
RNN CODE on GITHub
Awesome Recurrent Neural Networks
A curated list of resources dedicated to recurrent neural networks
Maintainers -
Myungsub Choi,
Jiwon Kim
pages for other topics:
awesome-deep-vision,
awesome-random-forest
https://github.com/kjw0612/awesome-rnn
Adversarial Attacks on AI APIs DNN online
Practical Black-Box Attacks against Deep Learning Systems using Adversarial Examples
http://arxiv.org/pdf/1602.02697v2.pdf
Feb 19, 2016
Nicolas Papernot - The Pennsylvania State University ngp5056@cse.psu.edu Patrick McDaniel - The Pennsylvania State University mcdaniel@cse.psu.edu Ian Goodfellow - Google Inc. goodfellow@google.com Somesh Jha - University of Wisconsin-Madison jha@cs.wisc.edu Z. Berkay Celik - The Pennsylvania State University zbc102@cse.psu.edu Ananthram Swami - US Army Research Laboratory ananthram.swami.civ@mail.mil
Abstract - Advances in deep learning have led to the
broad adoption of Deep Neural Networks (DNNs) to
a range of important machine learning problems, e.g.,
guiding autonomous vehicles, speech recognition, malware
detection. Yet, machine learning models, including
DNNs, were shown to be vulnerable to adversarial
samples—subtly (and often humanly indistinguishably)
modified malicious inputs crafted to compromise the integrity
of their outputs. Adversarial examples thus enable
adversaries to manipulate system behaviors. Potential
attacks include attempts to control the behavior
of vehicles, have spam content identified as legitimate
content, or have malware identified as legitimate software.
Adversarial examples are known to transfer from
one model to another, even if the second model has a
different architecture or was trained on a different set.
We introduce the first practical demonstration that this
cross-model transfer phenomenon enables attackers to
control a remotely hosted DNN with no access to the
model, its parameters, or its training data. In our demonstration,
we only assume that the adversary can observe
outputs from the target DNN given inputs chosen by the
adversary. We introduce the attack strategy of fitting a
substitute model to the input-output pairs in this manner,
then crafting adversarial examples based on this auxiliary
model. We evaluate the approach on existing DNN
datasets and real-world settings. In one experiment, we
force a DNN supported by MetaMind (one of the online
APIs for DNN classifiers) to mis-classify inputs at a rate
of 84.24%. We conclude with experiments exploring why
adversarial samples transfer between DNNs, and a discussion
on the applicability of our attack when targeting
machine learning algorithms distinct from DNNs.
Knowledge Vault: A Web-Scale Approach to Probabilistic Knowledge Fusion
https://www.cs.ubc.ca/~murphyk/Papers/kv-kdd14.pdf
Xin Luna Dong ∗ , Evgeniy Gabrilovich, Geremy Heitz, Wilko Horn, Ni Lao, Kevin Murphy † , Thomas Strohmann, Shaohua Sun, Wei Zhang Google, 1600 Amphitheatre Parkway, Mountain View, CA 94043 {lunadong|gabr|geremy|wilko|nlao|kpmurphy|tstrohmann|sunsh|weizh}@google.com
ABSTRACT
Recent years have witnessed a proliferation of large-scale knowledge bases, including Wikipedia, Freebase, YAGO, Microsoft’s Satori, and Google’s Knowledge Graph. To increase the scale even further, we need to explore automatic methods for constructing knowledge bases. Previous approaches have primarily focused on text-based extraction, which can be very noisy. Here we introduce Knowledge Vault, a Web-scale probabilistic knowledge base that combines extractions from Web content (obtained via analysis of text, tabular data, page structure, and human annotations) with prior knowledge derived from existing knowledge repositories. We employ supervised machine learning methods for fusing these distinct information sources. The Knowledge Vault is substantially bigger than any previously published structured knowledge repository, and features a probabilistic inference system that computes calibrated probabilities of fact correctness. We report the results of multiple studies that explore the relative utility of the different information sources and extraction methods.
Knowledge Vault Slides
http://www.slideshare.net/hustwj/kdd14-constructing-and-mining-webscale-knowledge-graphs
Published on Aug 25, 2014
Antoine Bordes (Facebook)
abordes@fb.com
Evgeniy Gabrilovich (Google)
gabr@google.com
A Review of “Knowledge Vault: A Web-Scale Approach to a Probabilistic Knowledge Fusion”
http://artent.net/2014/11/25/a-review-of-knowledge-vault-a-web-scale-approach-to-a-probabilistic-knowledge-fusion/
Deep Learning with TensorFlow
https://bigdatauniversity.com/courses/deep-learning-tensorflow/
This
Deep Learning with TensorFlow course focuses on TensorFlow. If you are new to the subject of deep learning, consider taking our
Deep Learning 101 course first.
Traditional neural networks rely on shallow nets, composed of one input, one hidden layer and one output layer. Deep-learning networks are distinguished from these ordinary neural networks having more hidden layers, or so-called more depth. These kind of nets are capable of discovering hidden structures within unlabeled and unstructured data (i.e. images, sound, and text), which consitutes the vast majority of data in the world.
TensorFlow is one of the best libraries to implement deep learning. TensorFlow is a software library for numerical computation of mathematical expressional, using data flow graphs. Nodes in the graph represent mathematical operations, while the edges represent the multidimensional data arrays (tensors) that flow between them. It was created by Google and tailored for Machine Learning. In fact, it is being widely used to develop solutions with Deep Learning.
In this TensorFlow course, you will be able to learn the basic concepts of TensorFlow, the main functions, operations and the execution pipeline. Starting with a simple “Hello Word” example, throughout the course you will be able to see how TensorFlow can be used in curve fitting, regression, classification and minimization of error functions. This concept is then explored in the Deep Learning world. You will learn how to apply TensorFlow for backpropagation to tune the weights and biases while the Neural Networks are being trained. Finally, the course covers different types of Deep Architectures, such as Convolutional Networks, Recurrent Networks and Autoencoders.
Course Syllabus
Module 1 – Introduction to TensorFlow
- HelloWorld with TensorFlow
- Linear Regression
- Nonlinear Regression
- Logistic Regression
- Activation Functions
Module 2 – Convolutional Neural Networks (CNN)
- CNN History
- Understanding CNNs
- CNN Application
Module 3 – Recurrent Neural Networks (RNN)
- Intro to RNN Model
- Long Short-Term memory (LSTM)
- Recursive Neural Tensor Network Theory
- Recurrent Neural Network Model
Module 4 - Unsupervised Learning
- Applications of Unsupervised Learning
- Restricted Boltzmann Machine
- Collaborative Filtering with RBM
Module 5 - Autoencoders
- Introduction to Autoencoders and Applications
- Autoencoders
- Deep Belief Network
GENERAL INFORMATION
- This TensorFlow course is free.
- This course if with Python language.
- It is self-paced.
- It can be taken at any time.
- It can be audited as many times as you wish.
RECOMMENDED SKILLS PRIOR TO TAKING THIS COURSE
REQUIREMENTS
