Elaticsearch学习
Elaticsearch
索引
1、索引创建
PUT /index_v1
{"settings": {"number_of_shards": 3,"number_of_replicas": 1},"mappings": {"properties": {"aaa": {"type": "keyword","store": true}, "hhh": {"type": "keyword","store": true}}}
}
2、索引别名
person_info_v1为索引名称，person_info为索引要创建的别名
put /person_info_v1/_alias/person_info
查询语法
1、minimum_should_match
bool查询也可以用 minimum_should_match， 如果配置成数字 3， 则表示 查询关键词被分词器分成 3 个及其以下的term 时， they are all required（条件都需要满足才能符合查询要求）
对于被analyzer分解出来的每一个term都会构造成一个should的bool query的查询,每个term变成一个term query子句。 例如"query": “how not to be”，被解析成： { “bool”: { “should”: [ { “term”: { “body”: “how”}}, { “term”: { “body”: “not”}}, { “term”: { “body”: “to”}}, { “term”: { “body”: “be”}} ],
2、查询分词效果
anlyzer后面是分词器，有ik_smart，ik_max_word等，text后面是想要查看分词效果的词
POST _analyze
{"analyzer":"ik_max_word","text":"李四"}
3、must和should混合使用
must是数据库中AND的意思，should是数据库中OR的意思，使用的时候不能简单的QueryBuilders.boolQuery.must().should()，要向下面这样使用
QueryBuilders.boolQuery().must(QueryBuilders.termQuery("is_deleted", DELETE_FLAG)).must(QueryBuilders.boolQuery().should(QueryBuilders.matchQuery("person_name", keywordVal).operator(Operator.AND).analyzer("ik_max_word") ));
Operato.AND表示查询分词要和es中的索引都匹配上才行，比如索引中内容是
张三三，分词效果是张和三三，查询内容是张三，分词是张和三，那这个时候就查询不到结果，查询内容改成张三三，分词效果是张和三三，就和索引中的分词都匹配上了，可以查询出内容。这样做的原因是防止你输入张三的时候把李三也查出来。如果不显示的声明Operator.AND，那会默认使用Operator.OR,这样的话输入张三，就会把李三也查出来，因为张三分词是张和三，只要三匹配了，就会查出来
4、查询索引中数据大小
GET /my-index-000001/_stats
5、字段匹配度排序
比如有个person_name字段，正常查询的时候按照_score排序，查询张建的时候，张建建的分值比张建的分值大，导致排序的时候张建建排在张建之前，但是按照常理来说，张建应该排在张建建之前，这就涉及到es的分词器以及分值计算问题了
解决方法是在person_name字段中设置一个子字段，不分词
"person_name": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart","store": true,"index_options": "docs","fields": {"raw": { "type": "keyword", "store": true }}}
查询的时候，使用match_parse精确查询子字段并用boost设置较大的权重，使用match模糊查询person_name字段
查询语句
1、短语匹配
{"query": {"bool": {"should": [{"match_phrase": {"person_name.raw": {"query": "张建建","boost": 10}}},{"match": {"person_name": {"query": "张建建"}}}]}}
}
java代码
BoolQueryBuilder queryBuilder = QueryBuilders.boolQuery();
queryBuilder.should(QueryBuilders.matchPhraseQuery("person_name.raw",keywordVal).boost(4));
queryBuilder.should(QueryBuilders.matchQuery("person_name", keywordVal).operator(Operator.AND).analyzer("ik_max_word"));
2、查询所有
/_search
{"query": {"match_all": {}}
}
3、查询数量
/_count
{"query": {"match_all": {}}
}
4、排序
{"query": {"match": {"ent_name": "杭州乾元"}},"sort": [{"est_date": {"order": "asc"}}]
}
5、nested查询
{"query": {"bool": {"filter": [{"nested": {"query": {"bool": {"filter": [{"term": {"clues.clue_id": {"value": "xxx","boost": 1}}}],"boost": 1}},"path": "clues","score_mode": "none","boost": 1}}],"boost": 1}}
}
6、字段+nested
{"query": {"bool": {"filter": [{"terms": {"_id": ["xxx"],"boost": 1}},{"nested": {"query": {"bool": {"filter": [{"terms": {"clues.clue_code": ["xxx"],"boost": 1}}],"adjust_pure_negative": true,"boost": 1}},"path": "clues","ignore_unmapped": false,"score_mode": "none","boost": 1}}],"adjust_pure_negative": true,"boost": 1}}
}
7、nested字段为空条件查询
{"query": {"bool": {"must_not": [{"nested": {"path": "tags","query": {"exists": {"field": "tags"}}}}]}}
}
8、案件数据为空，但是线索不为空的数据
{"query": {"bool": {"filter": [{"bool": {"should": [{"bool": {"must_not": [{"exists": {"field": "case_type"}}],"adjust_pure_negative": true,"boost": 1}}],"adjust_pure_negative": true,"boost": 1}},{"range": {"clue_num": {"from": "0","to": null,"include_lower": false,"include_upper": true,"boost": 1}}}]}}
}
删除
删除索引中的全部数据
POST /my_index/_delete_by_query
{"query": {"match_all": {}}
}
命令行删除：
curl -u elastic:'xxxx' -XPOST 'ip:port/medical_institution/_delete_by_query?refresh&slices=5&pretty' -H 'Content-Type: application/json' -d'{ "query": { "match_all": {} }}'
插入
POST /person_info_test_v1/_doc/
{"person_name": "张建芬"
}
更新
1、数据更新
（1）nested更新
POST http://ip:port/case_info/_update_by_query
{"script": {"source": "ctx._source.clues[0].clue_state = 2","lang": "painless"},"query": {"bool": {"filter": [{"nested": {"query": {"bool": {"filter": [{"term": {"clues.clue_id": {"value": "xxx","boost": 1}}}],"boost": 1}},"path": "clues","score_mode": "none","boost": 1}}],"boost": 1}}
}
（2）nested字段置空
{"script": {"source": "ctx._source.clues = []","lang": "painless"},"query": {"term": {"_id": "xxx"}}
}
（3）多条件更新
POST http://ip:port/case_info/_update_by_query
{"script": {"source": "ctx._source.obj_code = 'xxx'","lang": "painless"},"query": {"bool": {"filter": [{"term": {"case_type": "check_action"}},{"term": {"obj_code": "xxx"}}]}}
}
（4）数组(nested)字段更新
#更新为空的字段
{"script": {"source": "def tags= ctx._source.tags;def newTag=params.tagInfo; if (tags == null) { ctx._source.tags = params.tagInfo;}","lang": "painless","params": {"tagInfo": [{"tag_code": "case_xzcf_basic_0001","tag_value": "简易程序"},{"tag_code": "case_xzcf_basic_0002","tag_value": "立案阶段"},{"tag_code": "case_xzcf_basic_0003","tag_value": "无文书"}]}},"query": {"term": {"_id": "0e978d6afb74b52a322d7aa8fbfbddf8"}}
}
#将不为空的字段置为空
{"script": {"source": "def tags= ctx._source.tags;def newTag=params.tagInfo; ctx._source.tags = params.tagInfo;","lang": "painless","params": {"tagInfo": []}},"query": {"bool": {"must": [{"nested": {"path": "tags","query": {"exists": {"field": "tags"}}}}]}}
}
2、更新配置参数
PUT http://ip:port/case_info/_settings
{"refresh_interval": "1s"
}
访问
1、在linux中加密访问
#elastic是用户名，xxx是密码
curl ip:port -u elastic:'xxx'
2、ES健康状态查看
curl http://localhost:9200/_cat/health?v -u elastic:'xxx'
ES问题处理
一、数据插入失败
1、提示只读
] retrying failed action with response code: 403 ({"type"=>"cluster_block_exception", "reason"=>"index [person_info_v1] blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})
解决方法：首先查看磁盘空间是否被占满了，如果磁盘空间够用，则执行以下语句，将索引只读状态置为false
/indexname/_settings PUT
{"index": {"blocks": {"read_only_allow_delete": "false"}}
}{"index": {"refresh_interval": "1s"}
}
2、cpu占用过高
在网页上输入以下地址
http://ip:port/_nodes/hotthreads
问题处理
一、数据插入失败
1、提示只读
] retrying failed action with response code: 403 ({"type"=>"cluster_block_exception", "reason"=>"index [person_info_v1] blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})
解决方法：首先查看磁盘空间是否被占满了，如果磁盘空间够用，则执行以下语句，将索引只读状态置为false
/indexname/_settings PUT
{"index": {"blocks": {"read_only_allow_delete": "false"}}
}{"index": {"refresh_interval": "1s"}
}
2、cpu占用过高
在网页上输入以下地址
http://ip:port/_nodes/hotthreads
查询出的内容搜索cpu usage by thread即可