Papers with Code · CC BY-SA 4.0

Semantic Parsing

275 평가 기록 48 datasets

Semantic Parsing is the task of transducing natural language utterances into formal meaning representations. The target meaning representations can be defined according to a wide variety of formalisms. This include linguistically-motivated semantic representations that are designed to capture the meaning of any sentence such as λ-calculus or the abstract meaning representations. Alternatively, for more task-driven approaches to Semantic Parsing, it is common for meaning representations to represent executable programs such as SQL queries, robotic commands, smart phone instructions, and even general-purpose programming languages like Python and Java.

<span class="description-source">Source: Tranx: A Transition-based Neural Abstract Syntax Parser for Semantic Parsing and Code Generation </span>

개요는 각 항목의 일부입니다. 위 탭에서 전체 기록을 페이지별로 탐색할 수 있습니다.

벤치마크

Papers with Code 아카이브의 데이터셋별 기록 예시입니다. 평가 조건과 지표 방향이 검증되지 않아 순위를 의미하지 않습니다.

Dataset기록된 모델 예시MetricPaperCode
XiYan-SQL
Execution Accuracy % (Test): 75.63Execution Accuracy % (Dev): 73.34
논문 코드
StructBART + MBSE (IBM)
Smatch: 85.9
논문 코드
ARTEMIS-DA
Accuracy (Test): 80.8
논문 -
XiYan-SQL
Execution Accuracy (Test): 89.65
논문 코드
Graphene Smatch (MBSE paper) (IBM)
Smatch: 85.4
논문 코드
APT (IBM)
F1 Full: 78.5
논문 코드
RESDSQL-3B + NatSQL
Accuracy: 84.1
논문 코드
Spider-Agent + o1-preview
Success Rate: 17.03
논문 -
RASAT+PICARD
interaction match accuracy: 45.2question match accuracy: 67.7
논문 코드
Joint model
Smatch: 73.7
논문 코드
GPT-4 Turbo
TDEX: 67.0
논문 코드
Bi-LSTM seq2seq: BERT + characters in 1 encoder
F1: 88.3
논문 코드

데이터셋

48개

논문

라이브러리

Papers with Code 리더보드 row에 연결된 구현 저장소입니다.

하위 태스크