전체 글135 🌱ㅣAttention Attention어텐션 The seq2seq model, as mentioned earlier, relies only on a single fixed-size context vector at the end of the encoder, which can lead to information loss and a bottleneck problem. (If you are unsure about the content, please check the link below for another review.) 앞서 seq2seq 모델은 문제점으로 인코더 마지막에 있는 하나의 고정된 크기의 컨텍스트 벡터만 참고하니 정보 손실이 발생하면서 bottleneck 문제가 있다. (혹시 내용이 무엇인지 모르겠다면 아래 링크로 한번.. 2026. 2. 5. 🌱ㅣseq2seq (Sequence-to-Sequence) Seq2seq OverviewSeq2seq 개요 DNN(deep neural network) Nonlinear architectures and parallel computing have begun achieving state-of-the-art results across various fields. However, the standard DNN structures, which require fixed-dimensional inputs and outputs, pose limitations for problems that require inputs and outputs of non-fixed lengths (such as speech recognition and NLP). This constraint ari.. 2026. 2. 3. 🌱ㅣWord Embedding (2) ELMo Pre-trained language model techniques have become a key component in many NLP models, and a good word representation should model the following aspects well. Pre-trained language model 기법 같은 사전 훈련된 방식의 언어모델로 많은 NLP 모델에서 key component가 되어있는데 좋은 word representation은 아래 내용들을 잘 모델링해야 한다.- The grammatical and semantic properties of language use- Polysemy [e.g., bank / Bank Account (은행 계좌) vs. Ri.. 2026. 1. 30. 🌱ㅣWord Embedding (1) Pre-trained Word Embedding The embedding vector progresses in the order: word → integer encoding → embedding layer → embedding vector (dense vector). For example, using the word "Great", you can see the resulting vector as shown in the figure below. 임베딩 벡터는 어떤 단어 > 정수 인코딩 > 임베딩 층 통과 > 임베딩 벡터(Dense vector) 순서로 진행이 되며, Great 라는 단어로 예시를 들어보면 아래같은 사진으로 볼 수 있다. The Keras library provides an Embeddin.. 2026. 1. 28. 🌱ㅣWord Representation (2) Distributed Representation 분산표현 Distributed representations are based on the distributional hypothesis: when representing a word, the surrounding context is taken into account, so words that appear in similar contexts tend to have similar meanings. 분산 표현은 분포(분산) 가설에 기반하여 특정 단어를 표현할 때, 주변 단어들을 참고하여 단어를 표현하는 방법으로 비슷한 문맥에서 등장하는 단어들은 비슷한 의미를 가진다.Ex)▪ She is a beautiful woman.▪ She is an awesome woma.. 2026. 1. 24. 🌱ㅣWord Representation (1) Concept of Word Representation단어 표현의 개념 Word representation proceeds in the following order: collect and preprocess text data into integer indices through tokenization (splitting text into words or tokens) → remove stopwords → deduplicate → compute word frequencies and assign an index to each word (sentence-level processing uses integer encoding and padding). The question of how to represent tex.. 2026. 1. 23. 이전 1 2 3 4 5 6 7 ··· 23 다음
