Zhongyuan Zhu - 2015 - Evaluating Neural Machine Translation in English-Japanese Task

•

1 recomendación•484 vistas

Association for Computational Linguistics

WAT - W15-5007

Educación

Evaluating Neural Machine Translation
in English-Japanese Task
(TEAM ID: WEBLIO MT)
Zhongyuan Zhu
@raphaelshu
1

Empirically evaluate various models in EJ task
‣ Two network architectures
2
‣ Three recurrent units
‣ LSTM, GRU, IRNN
multi-layer encoder-decoder model soft-attention model
‣ Two kinds of training data
‣ naturally-ordered, pre-reordered

Results: evaluation scores
4
BLEU RIBES HUMAN JPO
Baseline phrase-based SMT 29.80 0.691
Baseline hierarchical phrase-based SMT 32.56 0.746
Baseline Tree-to-string SMT 33.44 0.758 30.00
Submitted system 1
(NMT)
34.19 0.802 43.50
Submitted system 2
(NMT + System combination)
36.21 0.809 53.75 3.81
Best competitor 1: NAIST
(Travatar System with NeuralMT Reranking)
38.17 0.813 62.25 4.04
Best competitor 2: naver
(SMT t2s + Spell correction + NMT reranking)
36.14 0.803 53.25 4.00

Finding & Insights
‣ Soft-attention models outperforms multi-layer
encoder-decoder models
‣ Training models on pre-reordered data hurts
the performance
‣ NMT models tend to make grammatically
valid but incomplete translations
5

Más contenido relacionado

La actualidad más candente

DL for sentence classification project Write-up

Hoàng Triều Trịnh

Multi-Layer Perceptrons

ESCOM

15 Machine Learning Multilayer Perceptron

Andres Mendez-Vazquez

Neural networks

HarshitGupta367

Introduction to Applied Machine Learning

SheilaJimenezMorejon

DL for setence classification project presentation

Hoàng Triều Trịnh

Machine Learning: Introduction to Neural Networks

Francesco Collova'

deep CNN vs conventional ML

Chao Han chaohan@vt.edu

https://telecombcn-dl.github.io/2017-dlsl/ Winter School on Deep Learning for Speech and Language. UPC BarcelonaTech ETSETB TelecomBCN. The aim of this course is to train students in methods of deep learning for speech and language. Recurrent Neural Networks (RNN) will be presented and analyzed in detail to understand the potential of these state of the art tools for time series processing. Engineering tips and scalability issues will be addressed to solve tasks such as machine translation, speech recognition, speech synthesis or question answering. Hands-on sessions will provide development skills so that attendees can become competent in contemporary data analytics tools.

Recurrent Neural Networks I (D2L2 Deep Learning for Speech and Language UPC 2...

Universitat Politècnica de Catalunya

Mca2050 computer architecture

smumbahelp

02 Fundamental Concepts of ANN

Tamer Ahmed Farrag, PhD

Artificial neural network

IshaneeSharma

2021 03-01-on the relationship between self-attention and convolutional layers

JAEMINJEONG5

Introduction to CNN

Shuai Zhang

Deep Belief Networks

Hasan H Topcu

Introduction For seq2seq(sequence to sequence) and RNN

Hye-min Ahn

Clustering tutorial

Lio Gonçalves

Artificial Neural Networks Lect5: Multi-Layer Perceptron & Backpropagation

Mohammed Bennamoun

Learning Financial Market Data with Recurrent Autoencoders and TensorFlow

Altoros

IOSR Journal of Computer Engineering (IOSR-JCE) is a double blind peer reviewed International Journal that provides rapid publication (within a month) of articles in all areas of computer engineering and its applications. The journal welcomes publications of high quality papers on theoretical developments and practical applications in computer technology. Original research papers, state-of-the-art reviews, and high quality technical notes are invited for publications.

A New Classifier Based onRecurrent Neural Network Using Multiple Binary-Outpu...

iosrjce

La actualidad más candente (20)

DL for sentence classification project Write-up

Multi-Layer Perceptrons

15 Machine Learning Multilayer Perceptron

Neural networks

Introduction to Applied Machine Learning

DL for setence classification project presentation

Machine Learning: Introduction to Neural Networks

deep CNN vs conventional ML

Recurrent Neural Networks I (D2L2 Deep Learning for Speech and Language UPC 2...

Mca2050 computer architecture

02 Fundamental Concepts of ANN

Artificial neural network

2021 03-01-on the relationship between self-attention and convolutional layers

Introduction to CNN

Deep Belief Networks

Introduction For seq2seq(sequence to sequence) and RNN

Clustering tutorial

Artificial Neural Networks Lect5: Multi-Layer Perceptron & Backpropagation

Learning Financial Market Data with Recurrent Autoencoders and TensorFlow

A New Classifier Based onRecurrent Neural Network Using Multiple Binary-Outpu...

Destacado

Language technology is rapidly evolving. A resurgence in the use of distributed semantic representations and word embeddings, combined with the rise of deep neural networks has led to new approaches and new state of the art results in many natural language processing tasks. One such exciting - and most recent - trend can be seen in multimodal approaches fusing techniques and models of natural language processing (NLP) with that of computer vision. The talk is aimed at giving an overview of the NLP part of this trend. It will start with giving a short overview of the challenges in creating deep networks for language, as well as what makes for a “good” language models, and the specific requirements of semantic word spaces for multi-modal embeddings.

Visual-Semantic Embeddings: some thoughts on Language

Roelof Pieters

Blockchain is a fundamental IT for secure value transfer over networks. For any asset registered in a cryptographic ledger, the whole Internet is a VPN for its confirmation, assurity, and transfer. Blockchain reinvents economics and governance for the digital age. The long-tail structure of digital networks allows personalized economic and governance services. Smartnetworks are a new form of automated global infrastructure for large-scale next-generation projects.

State of Blockchain 2017: Smartnetworks and the Blockchain Economy

Melanie Swan

Roee Aharoni - 2017 - Towards String-to-Tree Neural Machine Translation

Association for Computational Linguistics

Deep Learning & NLP: Graphs to the Rescue!

Roelof Pieters

iPhone5c的最后猜测

Yanbin Kong

Deep Qualia: Philosophy of Statistics, Deep Learning, and Blockchain Deep learning: What is it, why is it important, and what do I need to know? The aim of this talk is to discuss deep learning as an advanced computational method and its philosophical implications. Computing is a fundamental model by which we are understanding more about ourselves and the world. We think that reality is composed of patterns, which can be detected by machine learning methods. Deep learning is a complexity optimization technique in which algorithms learn from data by modeling high-level abstractions and assigning probabilities to nodes as they characterize the system and make predictions. An important challenge in deep learning is that these methods work in certain domains (image, speech, and text recognition), but we do not have a good explanation for why, which impedes a wider application of these solutions. Another recent advance in computational methods is blockchain technology which allows the secure transfer of assets and information, and the automated coordination of operations via a trackable remunerative ledger and smart contracts (automatically-executing Internet-based programs). This talk looks at how deep learning technology, particularly as coupled with blockchain systems, might be used to produce a new kind of global computing platform. The goal is for blockchain deep learning systems to address higher-dimensional computing challenges that require learning and dynamic response in domains such as economics and financial risk, epidemiology, social modeling, public health (cancer, aging), dark matter, atomic reactions, network-modeling (transportation, energy, smart cities), artificial intelligence, and consciousness.

Philosophy of Deep Learning

Melanie Swan

(Invited talk at Search Solutions 2015) A lot of recent work in neural models and “Deep Learning” is focused on learning vector representations for text, image, speech, entities, and other nuggets of information. From word analogies to automatically generating human level descriptions of images, the use of text embeddings has become a key ingredient in many natural language processing (NLP) and information retrieval (IR) tasks. In this talk, I will present some personal learnings from working on (neural and non-neural) text embeddings for IR, as well as highlight a few key recent insights from the broader academic community. I will talk about the affinity of certain embeddings for certain kinds of tasks, and how the notion of relatedness in an embedding space depends on how the vector representations are trained. The goal of this talk is to encourage everyone to start thinking about text embeddings beyond just as an output of a “black box” machine learning model, and to highlight that the relationships between different embedding spaces are about as interesting as the relationships between items within an embedding space.

Vectorland: Brief Notes from Using Text Embeddings for Search

Bhaskar Mitra

Chris Dyer - 2017 - CoNLL Invited Talk: Should Neural Network Architecture Re...

Association for Computational Linguistics

Economics, broadly defined, is concerned with the description and analysis of the production, distribution, and consumption of goods and services. Also related is how individuals and groups make choices about these goods and services, and the consequences of their decisions. Decisions might be explicitly in regard to money and resources, but the same principles pertain to any kind of decision. The general form of the problem is that wants are bigger than resources, and even if two choices are both free, there is an opportunity cost in terms of deploying resources or focus into one area and not another. The same structure of decision-making among multiple options, with there being an opportunity cost to the road not taken, may persist regardless of domain, whether in classical economics or distributed ledger economics.

Blockchain Economic Theory

Melanie Swan

Chenchen Ding - 2015 - NICT at WAT 2015

Association for Computational Linguistics

Satoshi Sonoh - 2015 - Toshiba MT System Description for the WAT2015 Workshop

Association for Computational Linguistics

Neural Models for Document Ranking

Bhaskar Mitra

Satoshi Sonoh - 2015 - Toshiba MT System Description for the WAT2015 Workshop

Association for Computational Linguistics

John Richardson - 2015 - KyotoEBMT System Description for the 2nd Workshop on...

Association for Computational Linguistics

Using Text Embeddings for Information Retrieval

Bhaskar Mitra

John Richardson - 2015 - KyotoEBMT System Description for the 2nd Workshop on...

Association for Computational Linguistics

Writing NodeJS applications is an easy task for JavaScript developers. However, getting what is happening under the hood in NodeJS may be intimidating, but understanding it is vital for web developers. Indeed, when you try to learn NodeJS, most tutorials are about the NodeJS ecosystem like Express, Socket.IO, PassportJS. It is really rare to see some tutorials about the NodeJS runtime itself. By this meetup, I want to spot the light on some advanced NodeJS topics so as to help developers answering questions an experienced NodeJS developer is expected to answer. Understanding these topics is essential to make you a much more desirable developer. I want to explore several topics including the famous event-loop along with NodeJS Module Patterns and how dependencies actually work in NodeJS. I hope that this meetup would help you to be more comfortable understanding advanced code written in NodeJS.

Advanced Node.JS Meetup

LINAGORA

Cs231n 2017 lecture13 Generative Model

Yanbin Kong

Search logs contain examples of frequently occurring patterns of user reformulations of queries. Intuitively, the reformulation "san francisco" → "san francisco 49ers" is semantically similar to "detroit" →"detroit lions". Likewise, "london"→"things to do in london" and "new york"→"new york tourist attractions" can also be considered similar transitions in intent. The reformulation "movies" → "new movies" and "york" → "new york", however, are clearly different despite the lexical similarities in the two reformulations. In this paper, we study the distributed representation of queries learnt by deep neural network models, such as the Convolutional Latent Semantic Model, and show that they can be used to represent query reformulations as vectors. These reformulation vectors exhibit favourable properties such as mapping semantically and syntactically similar query changes closer in the embedding space. Our work is motivated by the success of continuous space language models in capturing relationships between words and their meanings using offset vectors. We demonstrate a way to extend the same intuition to represent query reformulations. Furthermore, we show that the distributed representations of queries and reformulations are both useful for modelling session context for query prediction tasks, such as for query auto-completion (QAC) ranking. Our empirical study demonstrates that short-term (session) history context features based on these two representations improves the mean reciprocal rank (MRR) for the QAC ranking task by more than 10% over a supervised ranker baseline. Our results also show that by using features based on both these representations together we achieve a better performance, than either of them individually. Paper: http://research.microsoft.com/apps/pubs/default.aspx?id=244728

Exploring Session Context using Distributed Representations of Queries and Re...

Bhaskar Mitra

Deep Learning in practice : Speech recognition and beyond - Meetup

LINAGORA

Destacado (20)

Visual-Semantic Embeddings: some thoughts on Language

State of Blockchain 2017: Smartnetworks and the Blockchain Economy

Roee Aharoni - 2017 - Towards String-to-Tree Neural Machine Translation

Deep Learning & NLP: Graphs to the Rescue!

iPhone5c的最后猜测

Philosophy of Deep Learning

Vectorland: Brief Notes from Using Text Embeddings for Search

Chris Dyer - 2017 - CoNLL Invited Talk: Should Neural Network Architecture Re...

Blockchain Economic Theory

Chenchen Ding - 2015 - NICT at WAT 2015

Satoshi Sonoh - 2015 - Toshiba MT System Description for the WAT2015 Workshop

Neural Models for Document Ranking

Satoshi Sonoh - 2015 - Toshiba MT System Description for the WAT2015 Workshop

John Richardson - 2015 - KyotoEBMT System Description for the 2nd Workshop on...

Using Text Embeddings for Information Retrieval

John Richardson - 2015 - KyotoEBMT System Description for the 2nd Workshop on...

Advanced Node.JS Meetup

Cs231n 2017 lecture13 Generative Model

Exploring Session Context using Distributed Representations of Queries and Re...

Deep Learning in practice : Speech recognition and beyond - Meetup

Similar a Zhongyuan Zhu - 2015 - Evaluating Neural Machine Translation in English-Japanese Task

Long Zhou - 2017 - Neural System Combination for Machine Transaltion

Association for Computational Linguistics

Online learning techniques, such as Stochastic Gradient Descent (SGD), are powerful when applied to risk minimization and convex games on large problems. However, their sequential design prevents them from taking advantage of newer distributed frameworks such as Hadoop/MapReduce. In this session, we will take a look at how we parallelize parameter estimation for linear models on the next-gen YARN framework Iterative Reduce and the parallel machine learning library Metronome. We also take a look at non-linear modeling with the introduction of parallel neural network training in Metronome as well.

MLConf 2013: Metronome and Parallel Iterative Algorithms on YARN

Josh Patterson

Tokyo Webmining Talk1

Kenta Oono

Zhongyuan Zhu - 2015 - Evaluating Neural Machine Translation in English-Japan...

Association for Computational Linguistics

Presentation, navid khoob

Realsim, Fanavaran Sharif

A Platform for Accelerating Machine Learning Applications

NVIDIA Taiwan

Supervised sequence labelling with recurrent neural networks ch1 6

SungminYou

A Hybrid Method of CART and Artificial Neural Network for Short Term Load For...

Salford Systems

A seriously simple memetic approach with a high performance

Fabio Caraffini

Efficient Implementation of Self-Organizing Map for Sparse Input Data

ymelka

Ultrasound nerve segmentation, kaggle review

Eduard Tyantov

Extracted pages from Neural Fuzzy Systems.docx

dannyabe

6조

butest

Expert estimation from Multimodal Features

Xavier Ochoa

Tridiagonal solver in gpu

Hunan University

MaPU-HPCA2016

Shaolin Xie

Artificial neural networks have been adopted for a broad range of tasks in multimedia analysis and processing, such as visual and acoustic classification, extraction of multimedia descriptors or image and video coding. The trained neural networks for these applications contain a large number of parameters (weights), resulting in a considerable size. Thus, transferring them to a number of clients using them in applications (e.g., mobile phones, smart cameras) benefits from a compressed representation of neural networks. MPEG Neural Network Coding and Representation is the first international standard for efficient compression of neural networks (NNs). The standard is designed as a toolbox of compression methods, which can be used to create coding pipelines. It can be either used as an independent coding framework (with its own bitstream format) or together with external neural network formats and frameworks. For providing the highest degree of flexibility, the network compression methods operate per parameter tensor in order to always ensure proper decoding, even if no structure information is provided. The standard contains compression-efficient quantization and an arithmetic coding scheme (DeepCABAC) as core encoding and decoding technologies, as well as neural network parameter pre-processing methods like sparsification, pruning, low-rank decomposition, unification, local scaling, and batch norm folding. NNR achieves a compression efficiency of more than 97% for transparent coding cases, i.e. without degrading classification quality, such as top-1 or top-5 accuracies. This talk presents an overview of the context, technical features, and characteristics of the NN coding standard, and discusses ongoing topics such as incremental neural network representation.

Standardising the compressed representation of neural networks

Förderverein Technische Fakultät

Regular Expressions (RE) are widely used in many of today modern applications. Some of the most relevant uses are to find patterns among data, like in genomic markers research for DNA analysis, deep packet inspection, signature-based detection for network intrusion detection system or in search engine. In this talk we present a novel and efficient RE matching architecture for FPGAs, based on the concept of matching core. RE can be software-compiled into sequences of basic matching instructions that a matching core runs on input data, and can be replaced to change the RE to be matched. This architecture can easily scale up with the available resources and is customizable to multiple usage scenarios. In particular, TiRex can scale up to a multicore architecture and could be implemented on different type of FPGAs with not particular effort.

TiReX: Tiled Regular eXpression matching architecture

NECST Lab @ Politecnico di Milano

Intrusion detection systems (IDSs) are very important tools for providing information and computer security. In IDSs, the publicly available KDD’99, has been the most widely deployed data set used by researchers since 1999. Using a common data set has been provided to compare the results of different researches. The aim of this study is to find optimal methods of preprocessing the KDD’99 data set and employ the RBF learning algorithm to apply an Intrusion Detection System.

Protocol Type Based Intrusion Detection Using RBF Neural Network

Waqas Tariq

Forward neural network (FNN) execution relying on the algorithm of training and architecture selection. Different parameters using for nip out the architecture of FNN such as the connections number among strata, neurons hidden number in each strata hidden and hidden strata number. Feature architectural combinations exponential could be uncontrollable manually so specific architecture can be design automatically by using special algorithm which build system with ability generalization better. Determination of architecture FNN can be done by using the algorithm of optimization numerous. In this paper methodology new proposes achievement where FNN neurons respective with hidden layers estimation work where in this work collect algorithm training self organizing feature map (SOFM) with advantages to explain how the best architectural selected automatically by SOFM from criteria error testing based on architecture populated. Different size of dataset benchmark of 4 classifications tested for approach proposed.

Architecture neural network deep optimizing based on self organizing feature ...

journalBEEI

Similar a Zhongyuan Zhu - 2015 - Evaluating Neural Machine Translation in English-Japanese Task (20)

Long Zhou - 2017 - Neural System Combination for Machine Transaltion

MLConf 2013: Metronome and Parallel Iterative Algorithms on YARN

Tokyo Webmining Talk1

Zhongyuan Zhu - 2015 - Evaluating Neural Machine Translation in English-Japan...

Presentation, navid khoob

A Platform for Accelerating Machine Learning Applications

Supervised sequence labelling with recurrent neural networks ch1 6

A Hybrid Method of CART and Artificial Neural Network for Short Term Load For...

A seriously simple memetic approach with a high performance

Efficient Implementation of Self-Organizing Map for Sparse Input Data

Ultrasound nerve segmentation, kaggle review

Extracted pages from Neural Fuzzy Systems.docx

6조

Expert estimation from Multimodal Features

Tridiagonal solver in gpu

MaPU-HPCA2016

Standardising the compressed representation of neural networks

TiReX: Tiled Regular eXpression matching architecture

Protocol Type Based Intrusion Detection Using RBF Neural Network

Architecture neural network deep optimizing based on self organizing feature ...

Más de Association for Computational Linguistics

Muis - 2016 - Weak Semi-Markov CRFs for NP Chunking in Informal Text

Association for Computational Linguistics

Castro - 2018 - A High Coverage Method for Automatic False Friends Detection ...

Association for Computational Linguistics

Castro - 2018 - A Crowd-Annotated Spanish Corpus for Humour Analysis

Association for Computational Linguistics

Muthu Kumar Chandrasekaran - 2018 - Countering Position Bias in Instructor In...

Association for Computational Linguistics

Daniel Gildea - 2018 - The ACL Anthology: Current State and Future Directions

Association for Computational Linguistics

Elior Sulem - 2018 - Semantic Structural Evaluation for Text Simplification

Association for Computational Linguistics

Daniel Gildea - 2018 - The ACL Anthology: Current State and Future Directions

Association for Computational Linguistics

Wenqiang Lei - 2018 - Sequicity: Simplifying Task-oriented Dialogue Systems w...

Association for Computational Linguistics

Matthew Marge - 2017 - Exploring Variation of Natural Human Commands to a Rob...

Association for Computational Linguistics

Venkatesh Duppada - 2017 - SeerNet at EmoInt-2017: Tweet Emotion Intensity Es...

Association for Computational Linguistics

Chenchen Ding - 2015 - NICT at WAT 2015

Association for Computational Linguistics

Hyoung-Gyu Lee - 2015 - NAVER Machine Translation System for WAT 2015

Association for Computational Linguistics

Graham Neubig - 2015 - Neural Reranking Improves Subjective Quality of Machin...

Association for Computational Linguistics

Graham Neubig - 2015 - Neural Reranking Improves Subjective Quality of Machin...

Association for Computational Linguistics

Terumasa Ehara - 2015 - System Combination of RBMT plus SPE and Preordering p...

Association for Computational Linguistics

Terumasa Ehara - 2015 - System Combination of RBMT plus SPE and Preordering p...

Association for Computational Linguistics

Toshiaki Nakazawa - 2015 - Overview of the 2nd Workshop on Asian Translation

Association for Computational Linguistics

Hua Shan - 2015 - A Dependency-to-String Model for Chinese-Japanese SMT System

Association for Computational Linguistics

Wei Yang - 2015 - Sampling-based Alignment and Hierarchical Sub-sentential Al...

Association for Computational Linguistics

Wei Yang - 2015 - Sampling-based Alignment and Hierarchical Sub-sentential Al...

Association for Computational Linguistics

Más de Association for Computational Linguistics (20)