程序员的资源宝库

网站首页 > gitee 正文

ElasticSearch快速入门 elasticsearch7教程

sanyeah 2024-03-29 17:12:21 gitee 8 ℃ 0 评论

ElasticSearch快速入门

一、简介

ElasticSearch是一个基于Lucene的搜索服务器。它提供了一个分布式多用户能力的全文搜索引擎，基于RESTful web接口。Elasticsearch是用Java开发的，并作为Apache许可条款下的开放源码发布，是当前流行的企业级搜索引擎。设计用于云计算中，能够达到实时搜索，稳定，可靠，快速，安装使用方便。
其他搜索服务器：Solr
ElasticSearch与Solr区别：ElasticSearch对进行实时更新的数据进行搜索比较快，Solr对进行不经常更改的数据比较搜索比较快，遇到实时更新的数据会出现线程阻塞。
传统数据库为特定列增加一个索引，例如B-Tree索引来加速检索。Elasticsearch和Lucene使用一种叫做倒排索引(inverted index)的数据结构来达到相同目的，通俗点来说，倒排索引就是通过Value获取key，就是先把数据导入搜索服务器中，搜索服务器对数据进行分词，然后对分词进行归类，每一个文档（行）就声明一个词的出现频次、位置。

非必要最新版本，建议使用ElasticSearch7版本，ElasticSearch8版本对javaApI及java High Level Rest Cliet语法有所改变，但是java-client7.0与ElasticSearch8.0兼容，具体见官网:https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/installation.html

二、倒排索引

每一列表示字段值，每一行表示一个文档，每个类型相当于数据库中的数据表，但是在ElasticSearch7.0以后，为了提升查询效率，类型只是有`_doc`一种。

举例解析：比如谷歌这个词，文档频率是5，表示出现在5行中(见文档内容图)都有谷歌这个词,(1;1),(2;1)

,(3;2)...表示第一行出现一次，第二行出现一次，第三行出现二次，<1>表示出现在第一个单词位置上，<1;6>表示出现在第一个和第六个单词位置上，搜索服务器根据倒排列表的信息快速定位到单词位置，提升查询效率。

  • ElasticSearch与MySQL概率对比：

三、环境搭建(单机)

  • 下载软件(官网下载 选择相应的版本进行下载)

https://www.elastic.co/cn/downloads/elasticsearch

  • 把下载的Linux包上传Linux中相应的文件夹中
  • 解压压缩包
tar -zxvf elasticsearch-8.10.2-linux-x86_64.tar.gz

进入解压后的目录中 并修改elasticserach.yml

cd elasticsearch-8.10.2/config
vim elasticserach.yml

以下配置要启动服务器(cd elasticsearch-8.10.2/bin ./elasticsearch)后才能生成，生成以后就修改如下配置,外部浏览器才能访问或者直接添加xpack.security.enabled: false，记住开放端口号9200和9300。

配置Linux中允许创建的最大线程（root用户下）

vim /etc/security/limits.conf
  • 配置信息如下：
* soft nofile 65535
* hard nofile 65535
* soft nproc 4096
* hard nproc 4096

配置Linux中最大虚拟内存区域（root用户下）

vim /etc/sysctl.conf
  • 配置信息如下：
vm.max_map_count = 262144

重新加载虚拟内存（root用户下）

sysctl -p

注意要在非root用户下操作启动（root用户下）

groupadd es #创建用户组
useradd es -g es #创建用户赋予es用户组
chown es:es -Rf elasticsearch-8.10.2 #为新建的用户授权 防止权限不足无法启动
su es #切换用户启动服务器
cd elasticsearch-8.10.2/bin #进入启动目录
./elasticsearch  #启动服务器  （非root用户下操作）
./elasticsearch  -d  #守护进程后台启动（非root用户下操作）

测试访问

http://192.168.147.110:9200

四、搭建集群（三台服务器）

  • 注意事项
如果配置过单机版的es服务器，配置集群时，要重新解压安装包，否则容易出错，因为单机版es服务器启动后，会产生一些新的缓存和配置文件信息。
  • 上传文件并解压linux压缩包到三个不同服务器中（192.168.147.110;192.168.147.120;192.168.147.130）。
tar -zxvf elasticsearch-8.10.2-linux-x86_64.tar.gz
  • 进入解压文件中config，配置elasticSearch.yml文件(三台服务器要配置)
cluster.name: my-application #三个节点集群一样
node.name: node-2  #每个节点名不一样
network.host: 0.0.0.0 #所有的远程主机都可以访问
http.port: 9200 #端口号三个节点一样
discovery.seed_hosts: ["192.168.147.110", "192.168.147.120","192.168.147.130"] #配置三个主机IP
cluster.initial_master_nodes: ["node-1", "node-2","node-3"]  #配置集群节点
discovery.request_peers_timeout: 120s #节点与节点之间连接超时等待时间
http.cors.enabled: true #开启第三方工具可以连接
http.cors.allow-origin: "*"  #允许所有的第三方工具连接
xpack.security.enabled: false #关闭安全连接  否则单机 浏览器无法访问  集群会报错

配置Linux中允许创建的最大线程（root用户下）

vim /etc/security/limits.conf
  • 配置信息如下：
* soft nofile 65535
* hard nofile 65535
* soft nproc 4096
* hard nproc 4096

配置Linux中最大虚拟内存区域（root用户下）

vim /etc/sysctl.conf
  • 配置信息如下：
vm.max_map_count = 262144

重新加载虚拟内存（root用户下）

sysctl -p

分别启动三台不同服务器

groupadd es
useradd es -g es
chown es:es -Rf  /elasticsearch-8.10.2
su es #切换非root用户
cd /elasticsearch-8.10.2/bin #进入启动文件夹
./elasticsearch  #启动服务

查看集群的状态：访问 http://192.168.147.130:9200/_cluster/health?pretty

所有分片都正常，健康状态显示green，如果只是其中一个分片坏了，服务器还是能完整查询，如：node-1中2、3分片坏了，其他分别还是可以组件0,1,2,3,4完整查询，健康状态为yellow，如果分片坏了，无法组件完整数据，如：三个节点中所有的0坏了，无法排列组合完整，健康状态为red。

五、elasticSearch可视化工具安装

第三方插件

常用的第三方插件有：head、cerebro、elasticHD，这里只说明head插件安装。

安装git

yum -y install https://packages.endpointdev.com/rhel/7/os/x86_64/endpoint-repo.x86_64.rpm #安装endpoint镜像仓库  里面有新版git
yum install -y git
git --version  #验证安装

安装npm centOs7支持nodejs16.X及以下版本，版本太高可能安装失败！

  • 下载官方源：
#  root用户执行
curl -fsSL https://rpm.nodesource.com/setup_16.x | bash 

# 非root用户执行
curl -fsSL https://rpm.nodesource.com/setup_16.x | sudo bash -
  • 安装nodejs
yum install -y nodejs
  • 验证安装
node -v
npm -v
  • 更换镜像
npm config set registry https://registry.npmmirror.com/ #下载速度更快

安装head第三方插件

  • 进入head插件github开源项目站
https://github.com/mobz/elasticsearch-head
  • 配置三台主机允许跨域才能外部浏览器访问

  • git 拉取和启动elasticsearch-head插件
git clone https://github.com/mobz/elasticsearch-head.git #如果没网络去官网下载tag.gz文件再丢进服务器 然后解压进入项目根目录。 注意地址是https://开头
cd elasticsearch-head
npm install phantomjs-prebuilt --ignore-scripts   #这步要执行（重要）
npm install
npm run start
访问 http://192.168.147.110:9100/

出现以上界面表示集群搭建成功！

  • elasticSearch 基础语法
GET http://192.168.147.110:9200/test/_doc/1/  #查看id=1的值
put/post  http://192.168.147.110:9200/test/_doc/5/  #id值不存在就添加数据  id不设置  随机生成id
delete  http://192.168.147.110:9200/test/_doc/5/  #根据id删除

安装谷歌插件

六、安装Kibana可视化软件

一、进入官网下载：https://www.elastic.co/cn/downloads/kibana并上传到服务器

二、创建文件夹并解压

mkdir -p /usr/local/kibana  #c创建自定义文件夹
tar -zxvf kibana-*****-linux-x86_64.tar.gz -C /usr/local/kib

三、修改配置文件

vi config/kibana.yml
  • 配置文件如下：
# 服务端口，默认5601
server.port: 5601
# 允许访问IP
server.host: "0.0.0.0"
# 设置 elasticsearch 节点及端口
elasticsearch.hosts: ["http://192.168.147.110:9200", "http://192.168.147.120:9200",
"http://192.168.147.130:9200"]

四、启动kibana软件(进入kibana bin目录下 注意开放端口号)

./kibana --allow-root  #允许root用户启动
./kibana  #非root用户启动

五、访问请求路径(前提要启动es)

http://192.168.147.110:5601

七、安装IK分词器插件

找到官网下载：下载地址： https://github.com/medcl/elasticsearch-analysis-ik/releases将文件上传至服务器

方式一：创建ik目录，然后将ik分词器解压至ik目录(三个节点都需要操作)

方式二：在bin目录下执行命令来安装插件(三个节点都需要安装)

mkdir -p /usr/local/elasticsearch-/plugins/ik # 解压至ik目录

解压插件到 /usr/local/elasticsearch-***/plugins/ik/

unzip elasticsearch-analysis-ik-.zip -d /usr/local/elasticsearch-***/plugins/ik/

  • 注意：如果版本不匹配修改ik中的plugin-descriptor.properties配置
version=8.10.2  #要与elasticSearch版本一致
java.version=20 #要与elasticSearch加载的jdk版本一致
version=8.10.2 #要与elasticSearch版本一致

测试(在linux上)

curl -XPUT http://localhost:9200/index   #创建索引
#创建索引映射  text可以分词  key_word不分词  ik_smart：细粒度分词   ik_max_word：粗粒度分词
curl -XPOST http://localhost:9200/index/_mapping -H 'Content-Type:application/json' -d'
{
        "properties": {
            "content": {
                "type": "text",
                "analyzer": "ik_max_word",
                "search_analyzer": "ik_smart"
            }
        }
}'


#添加若干数据
curl -XPOST http://localhost:9200/index/_create/1 -H 'Content-Type:application/json' -d'
{"content":"美国留给伊拉克的是个烂摊子吗"}
'
curl -XPOST http://localhost:9200/index/_create/2 -H 'Content-Type:application/json' -d'
{"content":"公安部：各地校车将享最高路权"}
'
curl -XPOST http://localhost:9200/index/_create/3 -H 'Content-Type:application/json' -d'
{"content":"中韩渔警冲突调查：韩警平均每天扣1艘中国渔船"}
'
curl -XPOST http://localhost:9200/index/_create/4 -H 'Content-Type:application/json' -d'
{"content":"中国驻洛杉矶领事馆遭亚裔男子枪击 嫌犯已自首"}
'

#查询中国显示高亮
curl -XPOST http://localhost:9200/index/_search  -H 'Content-Type:application/json' -d'
{
    "query" : { "match" : { "content" : "中国" }},
    "highlight" : {
        "pre_tags" : ["<tag1>", "<tag2>"],
        "post_tags" : ["</tag1>", "</tag2>"],
        "fields" : {
            "content" : {}
        }
    }
}
'

八、mysql密码和授权

mysql -u root –p  #登录mysql成功后  执行以下操作
set password = password('root'); #修改密码  
SHOW VARIABLES LIKE 'validate_password%';  #展示密码当前修改策略


set global validate_password_policy=0; #设置密码安全策略 有三级:0表示low 1表示medium 2表示height
set global validate_password_length=4; #设置密码长度为4位

然后再按照对应的加密策略进行密码修改！

# 允许所有连接访问所有库的所有表(*.*)，连接密码为1234
GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '1234' WITH GRANT OPTION;
# 刷新权限
FLUSH PRIVILEGES;

九、安装Logstash（数据导入工具）

安装数据导入工具兼容多个数据库(mysql，oracle)，自动把数据库中的数据导入搜索服务器的索引库中。

下载并上传到虚拟机服务器并解压

mkdir -p /usr/local/logstash
tar -zxvf logstash-****.tar.gz -C /usr/local/logstash/

创建jdbc.conf配置文件在解压包根目录下

touch jdbc.conf
  • 配置信息如下：
input {
stdin {
         }
jdbc {
# 配置数据库信息
jdbc_conn句文件
statement_filection_string => "jdbc:mysql://localhost:3306/shop?useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai"
jdbc_driver_class => "com.mysql.cj.jdbc.Driver"
jdbc_user => "root"
jdbc_password => "root@123456"
jdbc_paging_enabled => "true"
jdbc_page_size => "50000"
jdbc_default_timezone => "Asia/Shanghai"
# 执行 sql 语epath => "/usr/local/logstash/logstash-8.10.2/jdbc.sql"
# 定时字段 各字段含义（由左至右）分、时、天、月、年，全部为*默认含义为每分钟都更
新
schedule => "* * * * *"
# 是否将 sql 中 column 名称转小写
lowercase_column_names => false
}
  }
output {

elasticsearch {
hosts => ["127.0.0.1:9200"]
index => "shop"
# 文档_id，%{goods_id}意思是取查询出来的goods_id的值，并将其映射到es的_id字段中
# 文档_id，%{goodsId}如果是别名，意思是取查询出来的goodsId的值，并将其映射到es的_id字段中
document_id => "%{goodsId}"
   } 
stdout {
codec => json_lines
}
 }

创建jdbc.sql 在解压包根目录下

SELECT
    goods_id goodsId,   -- 设置别名映射搜索服务器中的字段名
    goods_name goodsName,
    market_price marketPrice,
    original_img originalImg
FROM
    t_goods

把mysql-connection-8.0.*.jar包上传到../logstash-core/lib/jars目录中。

检查jdbc.conf配置文件是否可用

bin/logstash -f /usr/local/logstash/logstash-*.*.*/jdbc.conf -t

创建索引 （索引名为shop）

curl -X PUT http://localhost:9200/shop -H 'Content-Type:application/json' -d'{
"settings": {
"number_of_shards": 5,
"number_of_replicas": 1
  }
}'

创建字段分词器类型映射（设置goodsName字段名类型是text,使用ik分词器中的细粒度分词ik_max_word,也就是穷尽一切排列组合分词。）

ik_smart：粗粒度分词，比如中华人民共和国国歌，会拆分为中华人民共和国，国歌；
ik_max_word：细粒度分词，比如中华人民共和国国歌，会拆分为中华人民共和国，中华人民，中华，华人，人民共和国，人民，人，民，共和国，共和，和，国国，国歌，会穷尽各种可能的组

curl -XPOST http://localhost:9200/shop/_mapping -H 'Content-Type:application/json' -d'{
 "properties": {
 "goodsName": {
 "type": "text",
 "analyzer": "ik_max_word",
 "search_analyzer": "ik_max_word"
  }
   }
     }'

执行数据导入,执行时会和es保持联系，一直执行导入数据操作进行数据更新！

cd /usr/local/logstash/logstash-***/
bin/logstash -f /usr/local/logstash/logstash-*** /jdbc.conf #执行数据插入

十、Elasticsearch的JavaAPI

创建Maven基本工程

导入依赖

 <!--elasticsearch服务器-->
    <dependency>
      <groupId>org.elasticsearch</groupId>
      <artifactId>elasticsearch</artifactId>
      <version>7.17.13</version>
    </dependency>
  <!--client 客户端包-->
    <dependency>
      <groupId>org.elasticsearch.client</groupId>
      <artifactId>elasticsearch-rest-client</artifactId>
     <version>7.17.13</version>
   </dependency>
     <!--client 客户端依赖包-->
   <dependency>
      <groupId>org.elasticsearch.client</groupId>
     <artifactId>elasticsearch-rest-high-level-client</artifactId>
    <version>7.17.13</version>
   </dependency>

编写测试类(操作索引)

package com.zwf;

import org.apache.http.HttpHost;
import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest;
import org.elasticsearch.action.support.master.AcknowledgedResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.client.indices.CreateIndexRequest;
import org.elasticsearch.client.indices.CreateIndexResponse;
import org.elasticsearch.client.indices.GetIndexRequest;
import org.elasticsearch.client.indices.GetIndexResponse;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;

import java.io.IOException;

/**
 * Unit test for simple App.
 */
public class IndexTest {
   //默认是Http请求协议
    private static final HttpHost[] HTTPS={
            new HttpHost("192.168.147.110",9200),
            new HttpHost("192.168.147.120",9200),
            new HttpHost("192.168.147.130",9200),
    };
    private RestHighLevelClient client=null;
      @Before
    public void beforeEnv(){
        //版本一致时使用
       // client=new RestHighLevelClient(RestClient.builder(HTTPS));
        //高版本ElasticSearch 兼容低版本RestHighLevelClient使用，可以防止爆红异常信息。
        client= new RestHighLevelClientBuilder(RestClient.builder(HTTPS).build())
                .setApiCompatibilityMode(true)
                .build();
    }
    @After
    public void  afterEnv(){
         if (client!=null){
             try {
                 client.close();
             } catch (IOException e) {
                 throw new RuntimeException(e);
             }
         }
    }


    //创建索引
    @Test
    public void test() throws IOException {
        CreateIndexRequest index = new CreateIndexRequest("client_index");
        //发送请求
        CreateIndexResponse response = client.indices().create(index, RequestOptions.DEFAULT);
        boolean flag = response.isAcknowledged();
        //操作状态
        System.out.println("操作状态："+flag);
    }

    //查看索引
    @Test
    public void getIndex(){
        GetIndexRequest request=new GetIndexRequest("client_index");
        try {
            GetIndexResponse IndexResponse = client.indices().get(request, RequestOptions.DEFAULT);
            System.out.println("别名："+IndexResponse.getAliases());
            System.out.println("映射："+IndexResponse.getMappings());
            System.out.println("时间："+IndexResponse.getSettings());
        } catch (IOException e) {
            throw new RuntimeException(e);
        }

    }

    //删除索引
    @Test
    public void deleteIndex() throws IOException {
        DeleteIndexRequest del = new DeleteIndexRequest("client_index");
        AcknowledgedResponse delete = client.indices().delete(del, RequestOptions.DEFAULT);
        boolean b = delete.isAcknowledged();
        System.out.println(b);

    }


}

编写测试类（操作数据）

package com.zwf;

import org.apache.http.HttpHost;
import org.elasticsearch.action.DocWriteResponse;
import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest;
import org.elasticsearch.action.bulk.BulkRequest;
import org.elasticsearch.action.bulk.BulkResponse;
import org.elasticsearch.action.delete.DeleteRequest;
import org.elasticsearch.action.delete.DeleteResponse;
import org.elasticsearch.action.get.GetRequest;
import org.elasticsearch.action.get.GetResponse;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.action.update.UpdateRequest;
import org.elasticsearch.action.update.UpdateResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.common.text.Text;
import org.elasticsearch.index.engine.Engine;
import org.elasticsearch.index.query.QueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.rest.RestStatus;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightField;
import org.elasticsearch.search.fetch.subphase.highlight.Highlighter;
import org.elasticsearch.search.sort.SortBuilder;
import org.elasticsearch.search.sort.SortBuilders;
import org.elasticsearch.search.sort.SortOrder;
import org.elasticsearch.xcontent.XContent;
import org.elasticsearch.xcontent.XContentType;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;

import java.io.IOException;
import java.util.Arrays;
import java.util.HashMap;
import java.util.Map;
import java.util.Set;

/**
 * @author Mr Zeng
 * @version 1.0
 * @date 2023-10-05 11:32
 */
public class EsData_CRUD {

    //默认是Http请求协议
    private static final HttpHost[] HTTPS={
            new HttpHost("192.168.147.110",9200),
            new HttpHost("192.168.147.120",9200),
            new HttpHost("192.168.147.130",9200),
    };
    private RestHighLevelClient client=null;
    @Before
    public void beforeEnv(){
        client=new RestHighLevelClient(RestClient.builder(HTTPS));
    }

    @After
    public void  afterEnv(){
        if (client!=null){
            try {
                client.close();
            } catch (IOException e) {
                throw new RuntimeException(e);
            }
        }
    }

    @Test
    public void SaveData() throws IOException {
        Map<String,Object> jsons=new HashMap<>();
        jsons.put("username","password");
        jsons.put("age","18");
        jsons.put("sex","femal");
        IndexRequest index = new IndexRequest("client_index").id("1").source(jsons);
        IndexResponse response = client.index(index, RequestOptions.DEFAULT);
        DocWriteResponse.Result status = response.getResult();
        System.out.println(status.name());
    }
      @Test
    public void  QueryData() throws IOException {
        //指定索引库和id查询数据
        GetRequest index = new GetRequest("client_index", "1");
        GetResponse response = client.get(index, RequestOptions.DEFAULT);
        System.out.println(response.getSource());

    }

    @Test
    public void UpdateData() throws IOException {
        //声明要修改的文档
        Map<String,Object> map=new HashMap<>();
        map.put("username","zhangsan");
        map.put("age","30");
        map.put("sex","mal");
        //根据索引和id修改数据
       UpdateRequest index = new UpdateRequest("client_index","1").doc(map);
        UpdateResponse response = client.update(index, RequestOptions.DEFAULT);
        DocWriteResponse.Result result = response.getResult();
        System.out.println(result.getLowercase());
    }
      @Test
    public void deleteData() throws IOException {
          DeleteRequest clientIndex = new DeleteRequest("client_index", "1");
          DeleteResponse response = client.delete(clientIndex, RequestOptions.DEFAULT);
          System.out.println(response.status());
      }

    //批量增删改
    @Test
    public void makeBlur() throws IOException {
        BulkRequest request = new BulkRequest();
        request.add(
                new IndexRequest("client_index").id("1").source(XContentType.JSON,"username","root","passwd","123456","sex","female"),
                new IndexRequest("client_index").id("2").source(XContentType.JSON,"username","admin","passwd","1234785","sex","male"),
                new IndexRequest("client_index").id("3").source(XContentType.JSON,"username","administor","password","1872694955","sex","female"),
                new IndexRequest("client_index").id("4").source(XContentType.JSON,"username","admin","password","789456123","sex","male"));
        //批量更新
        request.add(new UpdateRequest("client_index","1").doc(XContentType.JSON,"username","black","password","987456","sex","male"));
        //批量删除
        request.add(new DeleteRequest("client_index","3"));
        BulkResponse bulk = client.bulk(request, RequestOptions.DEFAULT);
        System.out.println(bulk.hasFailures());

    }

    //批量查询
    @Test
    public void queryDoc() throws IOException {
        //指定搜 索引库 shop和index两个索引库
        SearchRequest searchRequest=new SearchRequest("shop","index");
        //构建查询对象
        SearchSourceBuilder searchSourceBuilder=new SearchSourceBuilder();
        //添加查询条件
        searchSourceBuilder.query(QueryBuilders.matchAllQuery());
        //执行请求
        searchRequest.source(searchSourceBuilder);
        SearchResponse search = client.search(searchRequest, RequestOptions.DEFAULT);
        //打印记录总数数值
        System.out.println(search.getHits().getTotalHits().value);
         //查询记录大于10条 默认显示10条  打印每个json对象  只能操作对象
        search.getHits().forEach(System.out::println);
        System.out.println("$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$");
         //获取每个对象的具体数值
        //结果值 查询记录大于10条 默认显示10条 可以操作对象的属性值
        SearchHit[] hits = search.getHits().getHits();
        for (SearchHit hit:hits){
            System.out.println(hit);  //每个对象
            System.out.println(hit.getScore());  //分数
            System.out.println(hit.getSourceAsMap());//内容
            System.out.println(hit.getIndex()); //索引库名
            System.out.println("******************************");
        }
    }
     //匹配查询
    @Test
    public void matchQuery() throws IOException {
    //指定查询的索引库
        SearchRequest request = new SearchRequest("index", "shop");
        //指定匹配查询条件
        SearchSourceBuilder sourceBuilder=new SearchSourceBuilder();
        //指定查询的关键字
        String keyWord="中国移动";
        //指定多条件匹配查询  goodName和content字段值进行了ik分词
        SearchSourceBuilder query = sourceBuilder.query(QueryBuilders.multiMatchQuery(keyWord, "goodsName", "content"));
        request.source(query);
        //进行查询操作
        SearchResponse search = client.search(request, RequestOptions.DEFAULT);
        //获取查询到的总记录数
        System.out.println("查询的记录数："+search.getHits().getTotalHits().value);
        //获取查询的具体属性值
        SearchHit[] hits = search.getHits().getHits();
        for (SearchHit hit:hits){
            System.out.println(hit.getIndex());
            System.out.println(hit.getScore());
            Map<String, Object> sourceAsMap = hit.getSourceAsMap();
            System.out.println("********************");
            Set<Map.Entry<String, Object>> entries = sourceAsMap.entrySet();
            for (Map.Entry<String,Object> entry:entries){
                System.out.println("查询内容："+entry.getValue());
            }
        }

    }
     //分页 排序查询
    @Test
    public void  getResultInfo() throws IOException {
        SearchRequest request=new SearchRequest("shop");
        SearchSourceBuilder builder=new SearchSourceBuilder();
        String keyword="中国移动联通";
        //goodsName字段值 进行分词
        builder.query(QueryBuilders.multiMatchQuery(keyword, "goodsName"));
        //进行score值升序排序
        builder.sort(SortBuilders.scoreSort().order(SortOrder.ASC));
        //进行分页  从0开始显示30条数据  分页是： 参数一：(currentPage-1)*pageSize  参数二：pageSize
        builder.from(0).size(30);
        request.source(builder);  //构建查询条件
        SearchResponse response = client.search(request, RequestOptions.DEFAULT);
        System.out.println("查询记录数："+response.getHits().getTotalHits().value);
        SearchHit[] hits = response.getHits().getHits();
        for (SearchHit hit:hits){
            System.out.println(hit.getScore());
            System.out.println(hit.getSourceAsMap());
            System.out.println("*************************");
        }

    }

    //高亮查询
    @Test
    public void highLightSearch() throws IOException {
        //指定查询索引库
        SearchRequest request = new SearchRequest("shop");
        //构建查询条件 高亮字段名和查询字段名
        SearchSourceBuilder builder=new SearchSourceBuilder();
        //显示0到29记录数
        builder.from(0).size(30);
        //构建查询关键字  goodsName进行了ik分词  细粒度分词
        String keyWord="中国移动联通";
        builder.query(QueryBuilders.multiMatchQuery(keyWord,"goodsName"));
        //构建高亮字段查询
        HighlightBuilder highlightBuilder = new HighlightBuilder();
        highlightBuilder.field("goodsName").preTags("<span color='red'>").postTags("<span/>");
        builder.highlighter(highlightBuilder);
        //构建根据价格降序排序
        builder.sort(SortBuilders.fieldSort("marketPrice").order(SortOrder.DESC));
        request.source(builder);
        SearchResponse search = client.search(request, RequestOptions.DEFAULT);
        long value = search.getHits().getTotalHits().value;
        System.out.println("查询记录数："+value);
        SearchHit[] hits = search.getHits().getHits();
        for (SearchHit hit:hits){
            hit.getSourceAsMap().forEach((s, o) -> {
                System.out.println(o);
            });
            System.out.println("******************");
            HighlightField goodsName = hit.getHighlightFields().get("goodsName");
            Text text = goodsName.fragments()[0];
            System.out.println("高亮字段html文本:"+text.string());
        }

    }

}

十一、springboot整合elasticSearch

导入依赖

      <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>

编写配置文件

spring:
  elasticsearch:
    uris: 192.168.147.110:9200,192.168.147.120:9200,192.168.147.130:9200

编写实体类，ORM映射elasticSearch中的索引对象

//createIndex = true默认自定创建索引分片5+5=10， createIndex = false手动创建索引在添加数据时创建索引
//如果在查询时  indexName="shop" 与elasticSearch索引库建立映射查询  如果插入数据时 会插入实体类指定的  索引中，没有索引就创建设置的索引。
@Document(indexName = "shop",shards=5,replicas = 5,createIndex = false)
public class Shop implements Serializable {


    @Id
    private Integer goodsId;

    /**
     * 商品名称  对产品名字进行细粒度分词  Text允许分词
     */
    @Field(type = FieldType.Text, analyzer = "ik_max_word")
    private String goodsName;

    /**
     * 市场价
     */
    @Field(type = FieldType.Double)
    private BigDecimal marketPrice;

    /**
     * 商品上传原始图  Keyword 不允许分词
     */
    @Field(type = FieldType.Keyword)
    private String originalImg;

    /**
     * t_goods
     */
    private static final long serialVersionUID = 1L;

    public Integer getGoodsId() {
        return goodsId;
    }

    public void setGoodsId(Integer goodsId) {
        this.goodsId = goodsId;
    }

    public String getGoodsName() {
        return goodsName;
    }

    public void setGoodsName(String goodsName) {
        this.goodsName = goodsName == null ? null : goodsName.trim();
    }

    public BigDecimal getMarketPrice() {
        return marketPrice;
    }

    public void setMarketPrice(BigDecimal marketPrice) {
        this.marketPrice = marketPrice;
    }

    public String getOriginalImg() {
        return originalImg;
    }

    public void setOriginalImg(String originalImg) {
        this.originalImg = originalImg == null ? null : originalImg.trim();
    }


    @Override
    public String toString() {
        StringBuilder sb = new StringBuilder();
        sb.append(getClass().getSimpleName());
        sb.append(" [");
        sb.append("Hash = ").append(hashCode());
        sb.append(", goodsId=").append(goodsId);
        sb.append(", goodsName=").append(goodsName);
        sb.append(", marketPrice=").append(marketPrice);
        sb.append(", originalImg=").append(originalImg);
        sb.append("]");
        return sb.toString();
    }


    public Shop() {
    }

    public Shop(Integer goodsId, String goodsName, BigDecimal marketPrice, String originalImg) {
        this.goodsId = goodsId;
        this.goodsName = goodsName;
        this.marketPrice = marketPrice;
        this.originalImg = originalImg;
    }
}

编写repositry接口类，根据泛型可以生成常用的增删改查的方法

//接口继承  第一个泛型是实体类  第二个泛型是id的数据类型 自动进行映射！
@Repository
public interface EsDataRepositry  extends ElasticsearchRepository<Shop, Integer> {

    List<Shop> findByGoodsName(String shopName);

   /**
     * 根据id查询商品
     * ？0为占位符
     * @param id
     * @return
     */
    @Query("{\"match\": {\"goodsId\":{ \"query\": \"?0\"}}}")
    Shop findByIdValue(Integer id);

}

编写测试类

@SpringBootTest
class SpringBootEsApplicationTests {
    //可以自定义方法对数据进行增删改查 跟数据库操作方法类似 注入继承ElasticsearchRepository子接口！
    @Autowired
    private EsDataRepositry esDataRepositry;
    //可以操作索引  匹配查询  复杂查询  显示高亮查询
    @Autowired
    private ElasticsearchRestTemplate template;
    @Test
    public void testSave(){
        //批量插入
       List<Shop> list=new ArrayList<>();
     list.add(new Shop(200,"测试数据",new BigDecimal("1563"),"1.jpg"));
      list.add(new Shop(210,"测试数据1",new BigDecimal("1780"),"2.jpg"));
       list.add(new Shop(220,"测试数据2",new BigDecimal("1630"),"3.jpg"));
       try {
           esDataRepositry.saveAll(list);
       }catch (Exception e){

      }

         //查询所有
        Iterable<Shop> all=esDataRepositry.findAll();
        all.forEach(System.out::println);

    }
     @Test
    public void  queryByName(){
        //根据商品名称查询信息
         List<Shop> byName = esDataRepositry.findByGoodsName("%中国%");
         byName.forEach(System.out::println);
     }

        //根据Id查询对象
      @Test
     public void quryById(){
        //配置repository层的自定义方法
          System.out.println(esDataRepositry. findByIdValue(100));

      }

      //创建索引
      @Test
      public void createIndex(){
        //设置实体类索引信息 返回索引操作对象
          IndexOperations indexOperations = template.indexOps(Shop.class);
          indexOperations.create();
          //创建索引映射
          Document mapping = indexOperations.createMapping();
          //将映射写入索引
          indexOperations.putMapping(mapping);
          //获取索引
          Map<String, Object> map = indexOperations.getMapping();
           map.forEach((k,v)->{
               System.out.println(k+"--------->"+v);

           });
           //索引是否存在
          boolean exists = indexOperations.exists();
          System.out.println(exists);
            //删除索引
          indexOperations.delete();
      }
        //操作文档  匹配查询
      @Test
      public void testDocument(){
        //删除指定索引和id 并返回Id
         String goodsId = template.delete("200", IndexCoordinates.of("shop1"));
         System.out.println(goodsId);
          //根据索引名和字段 匹配查询删除
        ByQueryResponse delete = template.delete(new NativeSearchQueryBuilder().withQuery(new MatchQueryBuilder("goodsName", "测试数据1")).build(), Shop.class, IndexCoordinates.of("shop1"));
          //id存在就更新  不存在就新增数据
          List<Shop> list=new ArrayList<>();
          list.add(new Shop(200,"测试数据2",new BigDecimal("1213"),"123.jpg"));
          list.add(new Shop(210,"测试数据3",new BigDecimal("89642"),"456.jpg"));
          template.save(list);
      }

       @Test
      //本地匹配查询
    public void testSearchMatch(){
        NativeSearchQueryBuilder queryBuilder = new NativeSearchQueryBuilder();
           NativeSearchQuery query = queryBuilder.withQuery(QueryBuilders.multiMatchQuery("中国移动联通电信", "goodsName")).build();
           SearchHits<Shop> search = template.search(query, Shop.class);
           search.forEach(shopSearchHit -> {
               System.out.println(shopSearchHit.getContent());
           });
       }
        //测试高亮效果 分页 排序
       @Test
       public void testHighLight(){
           NativeSearchQueryBuilder builder = new NativeSearchQueryBuilder();
             //构建查询条件
           NativeSearchQuery query = builder.withQuery(QueryBuilders.multiMatchQuery("中国", "goodsName"))
                   //根据市场价 降序显示  从0到4记录显示
                   .withPageable(Pageable.ofSize(5).withPage(0)).withSort(SortBuilders.fieldSort("marketPrice").order(SortOrder.DESC))
                   .withHighlightBuilder(new HighlightBuilder().field("goodsName").preTags("<span color='red'").postTags("</span>")).build();
           SearchHits<Shop> search = template.search(query, Shop.class);
           List<SearchHit<Shop>> searchHits = search.getSearchHits();
           searchHits.forEach(System.out::println);
           System.out.println("*******************");
           for (SearchHit<Shop> searchHit:searchHits){
                //结果集
               System.out.println(searchHit.getContent());
               //高亮信息
               System.out.println(searchHit.getHighlightField("goodsName").get(0));
               System.out.println("*******************");
           }
       }

}

springboot解决es-8.10.8高版本兼容低版本java high level client-7.17.13解决方案

  • 导入依赖
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.7.16</version>
        <relativePath/>
    </parent>

    <groupId>com.zwf</groupId>
    <artifactId>springBoot-es</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>springBoot-es</name>
    <description>springBoot-es</description>
    <properties>
        <java.version>1.8</java.version>
    </properties>
    <dependencies>

    <dependency>
        <groupId>org.springframework.data</groupId>
        <artifactId>spring-data-elasticsearch</artifactId>
        <version>4.4.16</version>
        <exclusions>
            <exclusion>
                <groupId>org.elasticsearch.client</groupId>
                <artifactId>transport</artifactId>
            </exclusion>
            <exclusion>
                <artifactId>transport-netty4-client</artifactId>
                <groupId>org.elasticsearch.plugin</groupId>
            </exclusion>
            <exclusion>
                <artifactId>elasticsearch-rest-high-level-client</artifactId>
                <groupId>org.elasticsearch.client</groupId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <artifactId>elasticsearch-rest-high-level-client</artifactId>
        <groupId>org.elasticsearch.client</groupId>
        <version>7.17.13</version>
    </dependency>
    <dependency>
        <groupId>org.elasticsearch.client</groupId>
        <artifactId>elasticsearch-rest-client</artifactId>
        <version>7.17.13</version>
    </dependency>
    <dependency>
        <artifactId>elasticsearch</artifactId>
        <groupId>org.elasticsearch</groupId>
        <version>7.17.13</version>
    </dependency>
    <dependency>
        <groupId>org.elasticsearch.plugin</groupId>
        <artifactId>transport-netty4-client</artifactId>
        <version>7.17.13</version>
    </dependency>
    <dependency>
        <groupId>org.elasticsearch.client</groupId>
        <artifactId>transport</artifactId>
        <version>7.17.13</version>
        <exclusions>
            <exclusion>
                <artifactId>elasticsearch</artifactId>
                <groupId>org.elasticsearch</groupId>
            </exclusion>
            <exclusion>
                <artifactId>transport-netty4-client</artifactId>
                <groupId>org.elasticsearch.plugin</groupId>
            </exclusion>
            <exclusion>
                <artifactId>elasticsearch-rest-client</artifactId>
                <groupId>org.elasticsearch.client</groupId>
            </exclusion>
        </exclusions>
    </dependency>


        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>
</project>

编写配置类(yml文件不要配置Elasticsearch配置了！)

//高版本ElasticSearch兼容低版本java-client
@Configuration
public class ElasticsearchConfig {
    //默认是Http请求协议
    private static final HttpHost[] HTTPS={
            new HttpHost("192.168.147.110",9200),
            new HttpHost("192.168.147.120",9200),
            new HttpHost("192.168.147.130",9200),
    };

        // 使用ES http客户端
        @Bean
        public RestHighLevelClient restHighLevelClient() {
            // IP以及port可以写入到application.yml配置文件中
            return new RestHighLevelClientBuilder(RestClient.builder(HTTPS).build())
                    .setApiCompatibilityMode(true)
                    .build();
        }
        // 引入ElasticsearchRestTemplate，方法名和Bean_name必须是elasticsearchTemplate
        @Bean
        public ElasticsearchRestTemplate elasticsearchTemplate() {
            return new ElasticsearchRestTemplate(restHighLevelClient());
        }

}

编写启动类（去除自动装配）

@SpringBootApplication(exclude = {ElasticsearchRepositoriesAutoConfiguration.class, ElasticsearchDataAutoConfiguration.class})
//扫描自定义实行继承的 ElasticsearchRepository<Shop,Integer> 子接口包
@EnableElasticsearchRepositories(value ="com.zwf.springbootes.Respositry")
public class SpringBootEsApplication {

    public static void main(String[] args) {
        SpringApplication.run(SpringBootEsApplication.class, args);
    }
}

自定义继承ElasticsearchRepository的接口 （在com.zwf.springbootes.Respositry包下）

@Repository
public interface EsDataRepositry  extends ElasticsearchRepository<Shop,Integer> {

    List<Shop> findByGoodsName(String shopName);

    /**
     * 根据id查询商品
     * ？0为占位符
     * @param id
     * @return
     */
    @Query("{\"match\": {\"goodsId\":{ \"query\": \"?0\"}}}")
    Shop findByIdValue(Integer id);

}

测试代码

@SpringBootTest
class SpringBootEsApplicationTests {
    @Autowired
    private EsDataRepositry esDataRepositry;
   @Resource
   private ElasticsearchRestTemplate elasticsearchTemplate;
    @Test
    public void testSave(){
        //批量插入
        List<Shop> list=new ArrayList<>();
        list.add(new Shop(200,"测试数据",new BigDecimal("1563"),"1.jpg"));
        list.add(new Shop(210,"测试数据1",new BigDecimal("1780"),"2.jpg"));
        list.add(new Shop(220,"测试数据2",new BigDecimal("1630"),"3.jpg"));

            esDataRepositry.saveAll(list);


         //查询所有
        Iterable<Shop> all=esDataRepositry.findAll();
        all.forEach(System.out::println);

    }
     @Test
    public void  queryByName(){
        //根据商品名称查询信息
         List<Shop> byName = esDataRepositry.findByGoodsName("%测试%");
         byName.forEach(System.out::println);
     }

        //根据Id查询对象
      @Test
     public void quryById(){
        //配置repository层的自定义方法
         System.out.println(esDataRepositry. findByIdValue(100));

      }

//      //创建索引
      @Test
     public void createIndex(){
       //设置实体类索引信息 返回索引操作对象
         IndexOperations indexOperations = elasticsearchTemplate.indexOps(Shop.class);
         indexOperations.create();
          //创建索引映射
          Document mapping = indexOperations.createMapping();
          //将映射写入索引
          indexOperations.putMapping(mapping);
          //获取索引
        Map<String, Object> map = indexOperations.getMapping();
          map.forEach((k,v)->{
             System.out.println(k+"--------->"+v);

           });
          //索引是否存在
         boolean exists = indexOperations.exists();
          System.out.println(exists);
           //删除索引
         indexOperations.delete();
     }
//        //操作文档  匹配查询
      @Test
      public void testDocument(){
//        //删除指定索引和id 并返回Id
         String goodsId = elasticsearchTemplate.delete("200", IndexCoordinates.of("shop1"));
        System.out.println(goodsId);
        //根据索引名和字段 匹配查询删除
        ByQueryResponse delete = elasticsearchTemplate.delete(new NativeSearchQueryBuilder().withQuery(new MatchQueryBuilder("goodsName", "测试数据1")).build(), Shop.class, IndexCoordinates.of("shop1"));
//          //id存在就更新  不存在就新增数据
         List<Shop> list=new ArrayList<>();
         list.add(new Shop(200,"测试数据2",new BigDecimal("1213"),"123.jpg"));
         list.add(new Shop(210,"测试数据3",new BigDecimal("89642"),"456.jpg"));
          elasticsearchTemplate.save(list);
    }
//
      @Test
//      //本地匹配查询
    public void testSearchMatch(){
        NativeSearchQueryBuilder queryBuilder = new NativeSearchQueryBuilder();
          NativeSearchQuery query = queryBuilder.withQuery(QueryBuilders.multiMatchQuery("中国移动联通电信", "goodsName")).build();
          SearchHits<Shop> search = elasticsearchTemplate.search(query, Shop.class);
          search.forEach(shopSearchHit -> {
              System.out.println(shopSearchHit.getContent());
         });
       }
//        //测试高亮效果 分页 排序
       @Test
       public void testHighLight(){
           NativeSearchQueryBuilder builder = new NativeSearchQueryBuilder();
             //构建查询条件
           NativeSearchQuery query = builder.withQuery(QueryBuilders.multiMatchQuery("中国", "goodsName"))
                   //根据市场价 降序显示  从0到4记录显示
                   .withPageable(Pageable.ofSize(5).withPage(0)).withSort(SortBuilders.fieldSort("marketPrice").order(SortOrder.DESC))
                   .withHighlightBuilder(new HighlightBuilder().field("goodsName").preTags("<span color='red'").postTags("</span>")).build();
           SearchHits<Shop> search = elasticsearchTemplate.search(query, Shop.class);
           List<SearchHit<Shop>> searchHits = search.getSearchHits();
           searchHits.forEach(System.out::println);
           System.out.println("*******************");
           for (SearchHit<Shop> searchHit:searchHits){
                //结果集
               System.out.println(searchHit.getContent());
               //高亮信息
               System.out.println(searchHit.getHighlightField("goodsName").get(0));
               System.out.println("*******************");
           }
       }


}

Tags：

猜你喜欢

本文暂时没有评论，来添加一个吧(●'◡'●)

欢迎 你 发表评论:

最近发表
标签列表