Shulou(Shulou.com)06/03 Report--

Cdn.com/01ee67ae04a648166d18ad84af8538056340b4f8.png ">

Author | Senior Development engineer of Yuanyi Aliyun Intelligent Business Group

Background

Message queuing for Apache Kafka is a distributed, high-throughput and scalable message queuing service provided by Aliyun. Message queuing for Apache Kafka is widely used in big data fields such as log collection, monitoring data aggregation, streaming data processing, online and offline analysis, and has become an indispensable part of big data's ecology.

Combined with the support of KafkaSource event source in Knative, it is convenient to interface with Kafka message service.

You can also install community Kafka version 2.0.0 and above.

Create Kafka instance on Ali Cloud and create Kafka instance

Log in to the message queuing Kafka console and select "purchase instance". Since the current Kafka event source in Knative supports version 2.0.0 or above, you need to purchase a prepaid or professional version to create an Kafka instance on Aliyun, and upgrade to 2.0.0 after purchase.

Deploy the instance and bind the VPC

After the purchase is completed, you can deploy and set the VPC where the Knative cluster resides:

Create Topic and Consumer Group

Next we create a Topic and a consumer group. Enter * * Topic Management * *, and click create Topic. Here we create a topic named demo:

Enter * * Consumer Group Management * *, and click create Consumer Group. Here we create a consumer group named demo-consumer:

Deploy Kafka data sources deploy Kafka addon components

Log in to the CCS console, go to "Knative component Management", and deploy Kafka addon components.

Create a KafkaSource instance

First, create a service event-display to receive events:

ApiVersion: serving.knative.dev/v1kind: Servicemetadata: name: event-displayspec: template: spec: containers:-image: registry.cn-hangzhou.aliyuncs.com/knative-sample/eventing-sources-cmd-event_display:bf45b3eb1e7fc4cb63d6a5a6416cf696295484a7662e0cf9ccdf5c080542c21d

Next, create the KafkaSource:

ApiVersion: sources.eventing.knative.dev/v1alpha1kind: KafkaSourcemetadata: name: alikafka-sourcespec: consumerGroup: demo-consumer # Broker URL. Replace this with the URLs for your kafka cluster, # which is in the format of my-cluster-kafka-bootstrap.my-kafka-namespace:9092. BootstrapServers: 192.168.0.6x:9092192.168.0.7x:9092192.168.0.8x:9092 topics: demo sink: apiVersion: serving.knative.dev/v1alpha1 kind: Service name: event-display

Description:

BootstrapServers: Kafka VPC access address consumerGroup: set consumer group topics: set Topic

After the creation is completed, we can check that the corresponding instance is running:

[root@iZ2zeae8wzyq0ypgjowzq2Z ~] # kubectl get podsNAME READY STATUS RESTARTS AGEalikafka-source-k22vz-db44cc7f8-879pj 1 Running 0 8h verification

In the Kafka console, select topic to send messages. Note that the message format here must be in json format:

We can see that the Kafka message sent has been received:

[root@iZ2zeae8wzyq0ypgjowzq2Z] # kubectl logs event-display-zl6m5-deployment-6bf9596b4f-8psx4 user-container ️CloudEvent: valid Context Attributes, SpecVersion: 0.2 Type: dev.knative.kafka.event Source: / apis/v1/namespaces/default/kafkasources/alikafka-source#demo ID: partition:7/offset:1 Time: 2019-10-18T08:50:32.492Z ContentType: application/json Extensions: key: demoTransport Context, URI: / Host: event-display.default.svc.cluster.local Method: POSTData {"key": "test"} Summary

Combined with Aliyun Kafka products, it is not simple and efficient to trigger the execution of services (functions) through event-driven. In this way, we can use Knative to fully release the native capabilities of the cloud, giving us more room for imagination. Welcome to communicate with those who are interested in Knative.

Practical video demonstration link: https://v.qq.com/x/page/d3013wzs3tp.html

About the author: yuan Yi, senior development engineer of Aliyun container platform, is responsible for the work related to Aliyun container platform Knative.

"Alibaba Yun × × icloudnative × × erverless, containers, Service Mesh and other technical fields, focusing on cloud native popular technology trends, cloud native large-scale landing practice, to do the best understanding of cloud native development × ×

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.