Struct SejongToken 

pub struct SejongToken {
    pub surface: String,
    pub pos: String,
    pub start_pos: usize,
    pub end_pos: usize,
    pub original_surface: Option<String>,
    pub original_pos: Option<String>,
}

세종 코퍼스 호환 토큰

Fields

surface: String

표면형

pos: String

세종 품사 태그

start_pos: usize

원본 텍스트 시작 위치

end_pos: usize

원본 텍스트 끝 위치

original_surface: Option<String>

분리 전 원본 표면형 (복합 형태소일 경우)

original_pos: Option<String>

분리 전 원본 품사 (복합 태그일 경우)

Implementations

impl SejongToken

pub fn new( surface: &str, pos: &str, start_pos: usize, end_pos: usize, ) -> SejongToken

새 세종 토큰 생성

pub fn from_split( surface: &str, pos: &str, start_pos: usize, end_pos: usize, original_surface: &str, original_pos: &str, ) -> SejongToken

분리된 토큰 생성 (원본 정보 포함)

pub fn to_sejong_format(&self) -> String

세종 형식 문자열 반환 (표면형/품사)

Trait Implementations

impl Clone for SejongToken

fn clone(&self) -> SejongToken

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SejongToken

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl PartialEq for SejongToken

fn eq(&self, other: &SejongToken) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Eq for SejongToken

impl StructuralPartialEq for SejongToken