Docker deploy Seata with Nacos

使用Docker部署Seata,同时使用Nacos作为配置中心,如果在你了解了Nacos和Seata之后开始捣鼓整一个Demo来学习一下,那么你一定是看了Seata的官网的,而且还很有可能是看了Github上的seata和seata-samples项目的,然后开始在本机上把 springcloud-nacos-seata 这个项目尝试跑起来,很遗憾你没能轻易如愿。我之前也是一样,现在把项目真正跑起来的过程整理一遍。

我们就以Seata官网的 一个微服务示例 架构图

然后参照 seata-samples项目的 springcloud-nacos-seata实例代码,做一个梳理。

目标:实现SpringCloud + Seata + Nacos + Mybatis + Druid + Mysql 整合。

说明:
(1)其中 Seata 和 Nacos 均采用Docker 部署,且是单机模式,准确来说是docker-compose部署。
(2)Seata 和 Nacos 版本均是最新版,Seata-1.2.0, Nacos-1.3.0.
Mysql 是5.7,我的用户名/密码:root/123456.
(3)Seata和Nacos是部署在两个不同的容器中,所以不要想着你的localhost,而要用你的宿主机ip (我的IP是192.168.1.2)

具体步骤:
(1)由于使用Nacos作为配置中心,所以第一步是启动nacos这个比较容易,直接使用下面standalone-derby.yaml文件,这个文件你最好不要直接copy,因为你会发现它挂载了其它文件。它来源于
https://github.com/nacos-group/nacos-docker example文件夹,所以你需要先下载这个仓库。至于下面的prometheus和grafana你可以暂且不管,如果你之前学习Nacos就知道了就更好。

version: "2"
services:
  nacos:
    image: nacos/nacos-server:latest
    container_name: nacos-standalone
    environment:
    - PREFER_HOST_MODE=hostname
    - MODE=standalone
    volumes:
    - ./standalone-logs/:/home/nacos/logs
    - ./init.d/custom.properties:/home/nacos/init.d/custom.properties
    ports:
    - "8848:8848"

  prometheus:
    container_name: prometheus
    image: prom/prometheus:latest
    volumes:
      - ./prometheus/prometheus-standalone.yaml:/etc/prometheus/prometheus.yml
    ports:
      - "9090:9090"
    depends_on:
      - nacos
    restart: on-failure

  grafana:
    container_name: grafana
    image: grafana/grafana:latest
    ports:
      - 3000:3000
    restart: on-failure

Nacos启动之后你就可以验证了:
http://localhost:8848/nacos/ 登录用户名和密码均是nacos

(2)由于Seata采用Nacos作为注册中心,所以在你启动Seata之前,你需要把配置写入到Nacos,当然Seata给你提供了一个shell脚本———nacos-config.sh,它来源于 https://github.com/seata/seata script文件夹,你只需要先配置好config.txt文件,然后一行命令就可以搞定。注意:config.txt和nacos-config.sh在同一目录下。原始的脚本在windows平台下可能需要修改一下,我会上传修改后的。

配置完成之后你可以在Nacos中验证了:找到->配置管理 -> 配置列表

当然上面是最后配置的效果,在此之前,我希望关注一下config.txt文件本身

transport.type=TCP
transport.server=NIO
transport.heartbeat=true
transport.enableClientBatchSendRequest=false
transport.threadFactory.bossThreadPrefix=NettyBoss
transport.threadFactory.workerThreadPrefix=NettyServerNIOWorker
transport.threadFactory.serverExecutorThreadPrefix=NettyServerBizHandler
transport.threadFactory.shareBossWorker=false
transport.threadFactory.clientSelectorThreadPrefix=NettyClientSelector
transport.threadFactory.clientSelectorThreadSize=1
transport.threadFactory.clientWorkerThreadPrefix=NettyClientWorkerThread
transport.threadFactory.bossThreadSize=1
transport.threadFactory.workerThreadSize=default
transport.shutdown.wait=3
transport.serialization=seata
transport.compressor=none
service.vgroupMapping.my_test_tx_group=default
service.default.grouplist=192.168.1.2:8091
service.enableDegrade=false
service.disableGlobalTransaction=false
client.rm.asyncCommitBufferLimit=10000
client.rm.lock.retryInterval=10
client.rm.lock.retryTimes=30
client.rm.lock.retryPolicyBranchRollbackOnConflict=true
client.rm.reportRetryCount=5
client.rm.tableMetaCheckEnable=false
client.rm.sqlParserType=druid
client.rm.reportSuccessEnable=false
client.rm.sagaBranchRegisterEnable=false
client.tm.commitRetryCount=5
client.tm.rollbackRetryCount=5
client.undo.dataValidation=true
client.undo.logSerialization=jackson
client.undo.logTable=undo_log
client.log.exceptionRate=100
store.mode=db
store.file.dir=file_store/data
store.file.maxBranchSessionSize=16384
store.file.maxGlobalSessionSize=512
store.file.fileWriteBufferCacheSize=16384
store.file.flushDiskMode=async
store.file.sessionReloadReadSize=100
store.db.datasource=druid
store.db.dbType=mysql
store.db.driverClassName=com.mysql.jdbc.Driver
store.db.url=jdbc:mysql://192.168.1.2:3306/seata?useUnicode=true
store.db.user=root
store.db.password=123456
store.db.minConn=5
store.db.maxConn=30
store.db.globalTable=global_table
store.db.branchTable=branch_table
store.db.queryLimit=100
store.db.lockTable=lock_table
store.db.maxWait=5000
server.recovery.committingRetryPeriod=1000
server.recovery.asynCommittingRetryPeriod=1000
server.recovery.rollbackingRetryPeriod=1000
server.recovery.timeoutRetryPeriod=1000
server.maxCommitRetryTimeout=-1
server.maxRollbackRetryTimeout=-1
server.rollbackRetryTimeoutUnlockEnable=false
server.undo.logSaveDays=7
server.undo.logDeletePeriod=86400000
metrics.enabled=false
metrics.registryType=compact
metrics.exporterList=prometheus
metrics.exporterPrometheusPort=9090

上面的配置是基于我的环境的,你可能需要做一些调整,我想强调几行分别是

(一)service.vgroupMapping.my_test_tx_group=default
说明my_test_tx_group 需要和代码中的tx-service-group一致

(二)store.mode=db
store.db.datasource=druid
store.db.url=jdbc:mysql://192.168.1.2:3306/seata?useUnicode=true
store.db.user=root
store.db.password=123456
说明上面采用db模式,Seata默认采用file模式,所以我们写db来覆盖,同时我们注意到url中有一个Schema是seata,没错我们一会需要在数据库中建好这个Schema同时我们注意到下面还有三张Table,我们一会一同建好。
store.db.globalTable=global_table
store.db.branchTable=branch_table
store.db.lockTable=lock_table

(三) 既然说到了数据库,那么我们顺便把实例微服务中的数据库也建好,主要是三个Schema,每一个Schema中都有一张业务表和undo_log表。可以先看一下

上面数据库的sql文件其实分为两个部分,一部分是Seata Server作为db模式所需要的seata库(branch_table,global_table,lock_table)及Seata Client (分别是account,order,storage)各自业务表及一张undo_log表。第一部分sql语句可以在Seata的下载文件中找到

第二部分可以在
https://github.com/seata/seata-samples/tree/master/springcloud-nacos-seata

(3)在数据库准备工作完毕后,我们使用docker-compose.yml启动Seata服务器

version: "3.1"

services:

  seata-server:
    image: seataio/seata-server:latest
    hostname: seata-server
    ports:
      - 8091:8091
    volumes:
      - ./config:/root/seata-config
    environment:
      - SEATA_PORT=8091
      - SEATA_IP=192.168.1.2
      - SEATA_CONFIG_NAME=file:/root/seata-config/registry

上面的配置中我们重点关注最后两行,一个是指定了IP,一个是指定了配置文件的位置。然后我们看一下配置文件——registry.conf。

registry {
  # file 、nacos 、eureka、redis、zk、consul、etcd3、sofa
  type = "nacos"

  nacos {
    serverAddr = "192.168.1.2"
    namespace = ""
    cluster = "default"
  }
  eureka {
    serviceUrl = "http://localhost:8761/eureka"
    application = "default"
    weight = "1"
  }
  redis {
    serverAddr = "localhost:6379"
    db = 0
    password = ""
    cluster = "default"
    timeout = 0
  }
  zk {
    cluster = "default"
    serverAddr = "127.0.0.1:2181"
    sessionTimeout = 6000
    connectTimeout = 2000
    username = ""
    password = ""
  }
  consul {
    cluster = "default"
    serverAddr = "127.0.0.1:8500"
  }
  etcd3 {
    cluster = "default"
    serverAddr = "http://localhost:2379"
  }
  sofa {
    serverAddr = "127.0.0.1:9603"
    application = "default"
    region = "DEFAULT_ZONE"
    datacenter = "DefaultDataCenter"
    cluster = "default"
    group = "SEATA_GROUP"
    addressWaitTime = "3000"
  }
  file {
    name = "file.conf"
  }
}

config {
  # file、nacos 、apollo、zk、consul、etcd3
  type = "nacos"

  nacos {
    serverAddr = "192.168.1.2"
    namespace = ""
    group = "SEATA_GROUP"
  }
  consul {
    serverAddr = "127.0.0.1:8500"
  }
  apollo {
    appId = "seata-server"
    apolloMeta = "http://192.168.1.204:8801"
    namespace = "application"
  }
  zk {
    serverAddr = "127.0.0.1:2181"
    sessionTimeout = 6000
    connectTimeout = 2000
    username = ""
    password = ""
  }
  etcd3 {
    serverAddr = "http://localhost:2379"
  }
  file {
    name = "file.conf"
  }
}

上面配置中
registry.type=nacos,
config.type=nacos,
nacos.serverAddr=”192.168.1.2″
nacos.group= “SEATA_GROUP” (这个和Nacos中相匹配)

(4)我们首先来看一下项目结构。

重点说明: 
(一)讲道理我们也是需要把account服务加上去的,但是用来验证分布式事务Seata的使用情况下,我们用两个就可以了。
(二)上面最主要的registry.conf文件和bootstrap.yml中spring.cloud.alibaba.seata.tx-service-group: ‘my_test_tx_group’(要和前面config.txt中 service.vgroupMapping.my_test_tx_group=default 相呼应)。
(三)再看registry.conf中最重要的部分:

registry {
  # file 、nacos 、eureka、redis、zk、consul、etcd3、sofa
  type = "nacos"
  nacos {
    application = "seata-server"
    serverAddr = "192.168.1.2"
    namespace = ""
    cluster = "default"
    username = "nacos"
    password = "nacos"
  }
  # 为了节省空间,其余type 省略
}

config {
  # file、nacos 、apollo、zk、consul、etcd3
  type = "nacos"

  nacos {
    application = "seata-server"
    serverAddr = "192.168.1.2"
    namespace = ""
    group = "SEATA_GROUP"
    username = "nacos"
    password = "nacos"
  }
  # 为了节省空间,其余type 省略
}

上面的application = “seata-server”和Nacos配置列表中保持一致

5.启动项目开始测试

(1)分布式事务成功,模拟正常下单、扣库存
localhost:9091/order/placeOrder/commit

(2)分布式事务失败,模拟下单成功、扣库存失败,最终同时回滚
localhost:9091/order/placeOrder/rollback

后记:这次整合Seata和Nacos的确花了一些时间,主要的原因是因为问题>人,自己能量级不够,所以这个问题才是问题,根本原因是因为没有理清思路,在众多配置文件和纷繁的零碎资料中走进了“问题森林”,会重点去查看一棵树,比如调试本地启动脚本jvm参数导致问题,配置文件中某项具体选项同时变化多个等等细节,而忘了初心是找到一个方向迅速走出这片森林。理思路,找方向,这件事一定要在头脑清晰的时候去做,头脑不清晰的时候只能算是“糊涂探索”。

最后希望这篇文章对你有帮助,我会把相关的脚本上传到我的github,欢迎clone.

Redis expire原理

https://redis.io/commands/expire

Available since 1.0.0.
Time complexity: O(1)

Set a timeout on key. After the timeout has expired, the key will automatically be deleted. A key with an associated timeout is often said to be volatile in Redis terminology.

The timeout will only be cleared by commands that delete or overwrite the contents of the key, including DELSETGETSET and all the *STORE commands. This means that all the operations that conceptually alter the value stored at the key without replacing it with a new one will leave the timeout untouched. For instance, incrementing the value of a key with INCR, pushing a new value into a list with LPUSH, or altering the field value of a hash with HSET are all operations that will leave the timeout untouched.

The timeout can also be cleared, turning the key back into a persistent key, using the PERSIST command.

If a key is renamed with RENAME, the associated time to live is transferred to the new key name.

If a key is overwritten by RENAME, like in the case of an existing key Key_A that is overwritten by a call like RENAME Key_B Key_A, it does not matter if the original Key_A had a timeout associated or not, the new key Key_A will inherit all the characteristics of Key_B.

Note that calling EXPIRE/PEXPIRE with a non-positive timeout or EXPIREAT/PEXPIREAT with a time in the past will result in the key being deleted rather than expired (accordingly, the emitted key event will be del, not expired).

Refreshing expires

It is possible to call EXPIRE using as argument a key that already has an existing expire set. In this case the time to live of a key is updated to the new value. There are many useful applications for this, an example is documented in the Navigation session pattern section below.

Differences in Redis prior 2.1.3

In Redis versions prior 2.1.3 altering a key with an expire set using a command altering its value had the effect of removing the key entirely. This semantics was needed because of limitations in the replication layer that are now fixed.

EXPIRE would return 0 and not alter the timeout for a key with a timeout set.

Return value

Integer reply, specifically:

  • 1 if the timeout was set.
  • 0 if key does not exist.

Examplesredis> SET mykey “Hello”

 redis> SET mykey "Hello"
"OK"
redis> EXPIRE mykey 10
(integer) 1
redis> TTL mykey
(integer) 10
redis> SET mykey "Hello World"
"OK"
redis> TTL mykey(integer)
-1
redis>

Pattern: Navigation session

Imagine you have a web service and you are interested in the latest N pages recently visited by your users, such that each adjacent page view was not performed more than 60 seconds after the previous. Conceptually you may consider this set of page views as a Navigation session of your user, that may contain interesting information about what kind of products he or she is looking for currently, so that you can recommend related products.

You can easily model this pattern in Redis using the following strategy: every time the user does a page view you call the following commands:

MULTI
RPUSH pagewviews.user:<userid> http://.....
EXPIRE pagewviews.user:<userid> 60
EXEC

If the user will be idle more than 60 seconds, the key will be deleted and only subsequent page views that have less than 60 seconds of difference will be recorded.

This pattern is easily modified to use counters using INCR instead of lists using RPUSH.

Appendix: Redis expires

Keys with an expire

Normally Redis keys are created without an associated time to live. The key will simply live forever, unless it is removed by the user in an explicit way, for instance using the DEL command.

The EXPIRE family of commands is able to associate an expire to a given key, at the cost of some additional memory used by the key. When a key has an expire set, Redis will make sure to remove the key when the specified amount of time elapsed.

The key time to live can be updated or entirely removed using the EXPIRE and PERSIST command (or other strictly related commands).

Expire accuracy

In Redis 2.4 the expire might not be pin-point accurate, and it could be between zero to one seconds out.

Since Redis 2.6 the expire error is from 0 to 1 milliseconds.

Expires and persistence

Keys expiring information is stored as absolute Unix timestamps (in milliseconds in case of Redis version 2.6 or greater). This means that the time is flowing even when the Redis instance is not active.

For expires to work well, the computer time must be taken stable. If you move an RDB file from two computers with a big desync in their clocks, funny things may happen (like all the keys loaded to be expired at loading time).

Even running instances will always check the computer clock, so for instance if you set a key with a time to live of 1000 seconds, and then set your computer time 2000 seconds in the future, the key will be expired immediately, instead of lasting for 1000 seconds.

How Redis expires keys

Redis keys are expired in two ways: a passive way, and an active way.

A key is passively expired simply when some client tries to access it, and the key is found to be timed out.

Of course this is not enough as there are expired keys that will never be accessed again. These keys should be expired anyway, so periodically Redis tests a few keys at random among keys with an expire set. All the keys that are already expired are deleted from the keyspace.

Specifically this is what Redis does 10 times per second:

  1. Test 20 random keys from the set of keys with an associated expire.
  2. Delete all the keys found expired.
  3. If more than 25% of keys were expired, start again from step 1.

This is a trivial probabilistic algorithm, basically the assumption is that our sample is representative of the whole key space, and we continue to expire until the percentage of keys that are likely to be expired is under 25%

This means that at any given moment the maximum amount of keys already expired that are using memory is at max equal to max amount of write operations per second divided by 4.

How expires are handled in the replication link and AOF file

In order to obtain a correct behavior without sacrificing consistency, when a key expires, a DEL operation is synthesized in both the AOF file and gains all the attached replicas nodes. This way the expiration process is centralized in the master instance, and there is no chance of consistency errors.

However while the replicas connected to a master will not expire keys independently (but will wait for the DEL coming from the master), they’ll still take the full state of the expires existing in the dataset, so when a replica is elected to master it will be able to expire the keys independently, fully acting as a master.


上面的内容来自于redis官网,其实如果我们在java代码中也是可以有所发现的,下面是Jedis的expire() 及API.

public Long expire(final String key, final int seconds) {
	checkIsInMulti();
	client.expire(key, seconds);
	return client.getIntegerReply();
}

Set a timeout on the specified key. After the timeout the key will be automatically deleted by the server. A key with an associated timeout is said to be volatile in Redis terminology.
Voltile keys are stored on disk like the other keys, the timeout is persistent too like all the other aspects of the dataset. Saving a dataset containing expires and stopping the server does not stop the flow of time as Redis stores on disk the time when the key will no longer be available as Unix time, and not the remaining seconds.
Since Redis 2.1.3 you can update the value of the timeout of a key already having an expire set. It is also possible to undo the expire at all turning the key into a normal key using the PERSIST command.
Time complexity: O(1)

Integer reply, specifically: 1: the timeout was set. 0: the timeout was not set since the key already has an associated timeout (this may happen only in Redis versions < 2.1.3, Redis >= 2.1.3 will happily update the timeout), or the key does not exist.

JPA根据某个对象的集合属性是否包含某个特定对象查询

标题尽管有点长,但是确实意义明确的。可以先看一下两个简化后对象。Video 和 Star,它们的关系是一部Video有多个Star,这里的Star不是github上的star,而是actor or actress。

@Data
@Document
public class Video{
    @Id
    protected String id;
    @DBRef
    private Set<Star> stars = new LinkedHashSet<>();
}

@Data
@Document
public class Star{
    @Id
    protected String id;
    @Indexed
    private String name;
}

现在有个查询需求,根据Star的id去查询他(她)的所有参演Video.目前我们的思路是:
(1)根据id查询出某个Star star;
(2)然后查询所有的Video,然后遍历的时候用stars.contains(star)晒选;
(3)筛选的结果就是该Star所参演的Video.

public List<Video> findAllByStarId(String starId) {
        Star star = starService.findById(starId);
        List<Video> all = repo.findAll();
        if (CollectionUtils.isEmpty(all)) {
            return Collections.emptyList();
        }
        return all.stream()
                .filter(t -> t.getStars().contains(star))
               .sorted(Comparator.comparing(Video::getRelease).reversed())
               .collect(Collectors.toList());
}

就功能型,上面是满足的,但是你实际运行起来就会发现查询很慢,估计20-30s,显然不符合我们的预期。

所有上面的代码有一个致命的隐患,那就是List all = repo.findAll();这是一个超级大隐患,查询的时候一定不能一次查询所有的数据,然后很有可能程序 一下子就卡了,这个和数据的多少及对象大小有直接关系。在生产环境绝对不能这么写代码的。

那么JPA有没有为我们提供一个比较好的方式。 当然是有的,那就是把我们在后面filter(t -> t.getStars().contains(star))这段代码直接在查询的时候就搞定。

public interface VideoRepository extends MongoRepository<Video, BigInteger> {
    
    List<Video> findAllByStarsContains(Star star, Sort sort);
    
}

修改后的代码是下面这样的:

    public List<Video> findAllByStarId(String starId) {
        Star star = starService.findById(starId);
        List<Video> all = repo.findAllByStarsContains(star, Sort.by(Sort.Direction.DESC, "release"));
        return CollectionUtils.isEmpty(all) ? Collections.emptyList() : all;
}

后记:
(1) 如果我们建模的时候是双向建模,Star 也有多部Video到还可以一次可以查询出来,可能我们由于数据原因,没有建立这样的关联,再讲白一点,就是Star只是包含一个简单的属性(name),只是为了扩展,慢慢有了birth,height…属性独立出去了。
(2) JPA的强大之处就是帮省去了写实现的时间,你只要按照规范命名就可以了。至于规范,在IDEA中的提示真的很友好。

You worth it.

Posted in JPA