Shulou(Shulou.com)06/01 Report--

What is the analysis and solution of ActiveMQ problem? in view of this problem, this article introduces the corresponding analysis and solution in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible method.

1) how to choose KahaDb and AMQ Message Store persistence methods?

Official:

From 5.3onwards-we recommend you use KahaDB-which offers improved scalability and recoverability over the AMQ Message Store.

The AMQ Message Store which although faster than KahaDB-does not scales as well as KahaDB and recovery times take longer.

Unofficial:

The kaha file system is actually a file index system, which consists of two parts, one is the data file system, which is composed of independent files, the default file size is 32m (configurable), and the other is the index file system, which records the location information of the message in the data file and the free block information in the data file. The data files are stored on the hard disk and the index files are cached in memory. So this storage system is beneficial to the storage of big messages. Text messages like our memberId are actually a waste. The index is larger than the message, ha.

Our analysis:

Recommended: Amq persistence

Reason: although KahaDB persistence is officially recommended, the advantages mentioned: good scalability and resilience are of little significance to our practical application. From our own experience, KahaDB persistence, Data file is a large file (feel that the file is too large, the possibility of causing queue service paralysis will increase), from the relevant configuration of the official website (Appendix 1) can not find where to set the maximum Size of the data file. The Amq persistence method can set the maximum Size of Data files, which ensures that there is a large backlog of instant messages and the Data file is not too large.

2) error: Channel was inactive for too long

Solution:

Add: wireFormat.maxInactivityDuration=0 to the Uri where the connection is established

Reference Resources:

Http://jinguo.iteye.com/blog/243153

You can do the following to fix the issues:

1) Append max inactivity duration to your Uri in the format below: wireFormat.maxInactivityDuration=0

2) Use the same Uri at the client side as well as at the server side

Regards

If you do not set this, the corresponding error will occur:

2008-05-07 09 Async exception with no exception listener 22 Async exception with no exception listener 56 343 [org.apache.activemq.ActiveMQConnection]-[WARN] Async exception with no exception listener: org.apache.activemq.transport.InactivityIOException: Channel was inactive for too long: localhost/127.0.0.1:61616

Org.apache.activemq.transport.InactivityIOException: Channel was inactive for too long: localhost/127.0.0.1:61616

Add a parameter such as? wireFormat.maxInactivityDuration=0 after the tcp url:tcp://localhost:61616 of ActiveMQ, otherwise a "Channel was inactive for too long" exception will be thrown when there is no message sent for a period of time.

3) error: Wire format negotiation timeout: peer did not send his wire format.

Solution:

1) close ActiveMqLog4j

Open: conf/log4j.properties

Will: log4j.rootLogger=INFO, console, logfile

Modified to: log4j.rootLogger=OFF

2) add: maxInactivityDurationInitalDelay=30000 to the Uri where the connection is established

For example, the test environment in Beijing is connected to Uri:

Tcp://192.168.199.80:61616?wireFormat.maxInactivityDuration=0&maxInactivityDurationInitalDelay=30000&connection.AsyncSend=true

Reference Resources:

Http://activemq.apache.org/javaxjmsjmsexception-wire-format-negociation-timeout-peer-did-not-send-his-wire-format.html

If you get exception like this,it can mean one of three things:

1. You're connecting to the port not used by ActiveMQ TCP transport

Make sure to check that you're connecting to the appropriate host:port

2. You're using log4j JMS appender and doesn't filter out ActiveMQ log messages

Be sure to read How do I use log4j JMS appender with ActiveMQ and more importantly to never send ActiveMQ log messages to JMS appender

3. Your broker is probably under heavy load (or network connection is unreliable), so connection setup cannot be completed in a reasonable time

If you experience sporadic exceptions like this, the best solution is to use failover transport, so that your clients can try connecting again if the first attempt fails. If you're getting these kind of exceptions more frequently you can also try extending wire format negotiation period (default 10 sec). You can do that by using wireFormat.maxInactivityDurationInitalDelay property on the connection URL in your client.

For example

Tcp://localhost:61616?wireFormat.maxInactivityDurationInitalDelay=30000

Will use 30 sec timeout. (there seems to be a problem! )

4) error: Out of memory

Solution:

1) set the maximum memory limit of Java to the appropriate size:

ACTIVEMQ_OPTS=-Xmx512M in Bin/activemq.bat (default is 512)

2) Activemq.xml configuration section: the systemUsage/ systemUsage configuration size is appropriate, and pay special attention to: greater than the sum of the memoryUsage of all durable desitination settings.

Note:

1) Angle brackets: ">" represents a wildcard

2) configuration of ACTIVEMQ_OPTS > = configuration in memoryUsage > = sum of all durable desitination settings

3) the SystemUsage configuration sets some system memory and hard disk capacity. When the system consumption exceeds these capacity settings, amq will "slow down producer", which is still very important.

Reference:

Http://m.oschina.net/blog/26216

Reference-http://activemq.apache.org/javalangoutofmemory.html

The content utility for MQ is manageable and configurable. The first thing to determine is which part of the MQ system leaks due to insufficient memory, the JVM,broker or the consumer or producer?

I. memory management

JVM memory Management:

1. Run broker in a stand-alone JVM with the bin/activemq command. Use the-Xmx and-Xss commands (modify the ACTIVEMQ_OPTS option parameter in the activemq.bat file)

two。 By default, MQ uses 512m JVM

Broker memory Management:

1. The memory used by broker is not determined by the memory of JVM. Although limited by JVM, broker does manage memory independently

2. The memory limits of systemUsage and destination are closely related to broker memory.

3. The relationship of memory in MQ is: JVM- > Broker- > broker features.

4. The total memory of all destination cannot exceed the total memory of broker

Consumers:

1. Because the message size is configurable, prefetch limit is often the main cause of memory overflow

two。 Reducing the size of the prefetch limit reduces the number of messages stored in the consumer's memory

Producer:

1. Unless the number of messages exceeds the broker resource limit, the producer will not cause a memory overflow

two。 When memory overflows, the producer will receive a blocking message from broker

II. Other

Hard disk that buffers messages:

1. Fast matching and distribution of messages are allowed only when messages are stored in memory, and memory may be exhausted when consumers are slow or leave

two。 When destination reaches its memory threshold, broker uses message cursors to cache non-persistent messages to the hard disk.

3. The critical value is configured in broker through memoryUsage and systemUsage attributes. Please refer to activemq.xml

4. For slow consumers, this feature allows producers to continue sending messages to broker before running out of memory or switching to producer concurrency control mode

5. When there is more than one destination, the default memory threshold may be broken, and it makes sense to cache messages on the hard disk.

6. PrecentUsage configuration: use percentage to control memory usage

Multiple threads:

1. By default, each destination of MQ corresponds to a unique thread

2.-Dorg.apache.activema.UseDedicatedTaskRunner=false (modify the ACTIVEMQ_OPTS option parameter in the activemq.bat file), and use thread pool to limit the number of threads, thereby reducing memory consumption

Big data transmits:

1. Destination policies--maxPageSize: controls the number of messages entering memory; lazyDispatch: increases the prefetch value that controls the use of the current consumer list

two。 Use blogMessage or streamsMessage types to transfer a large number of files

Leak JMS resources:

1. When there are a large number of session or producer or consumer without shutting down

two。 Use PooledConnectionFactory

5) jam caused by failover connection

Solution:

No failover connection

Analysis:

Using failover mode to connect, if the server or Activemq service you want to connect to is down, then the program will always be in a waiting state, no timeout, no error.

II) Appendix 1) KahaDB persistent configuration

Property name

Default value

Comments

Directory

Activemq-data

The path to the directory to use to store the message store data and log files

IndexWriteBatchSize

one thousand

Number of indexes written in a batch

IndexCacheSize

10000

Number of index pages cached in memory

EnableIndexWriteAsync

False

If set, will asynchronously write indexes

JournalMaxFileLength

32mb

A hint to set the maximum size of the message data logs

EnableJournalDiskSyncs

True

Ensure every non transactional journal write is followed by a disk sync (JMS durability requirement)

CleanupInterval

30000

Time (ms) before checking for a discarding/moving message data logs that are no longer used

CheckpointInterval

5000

Time (ms) before checkpointing the journal

IgnoreMissingJournalfiles

False

If enabled, will ignore a missing message log file

CheckForCorruptJournalFiles

False

If enabled, will check for corrupted Journal files on startup and try and recover them

ChecksumJournalFiles

False

Create a checksum for a journal file-to enable checking for corrupted journals

Available since version 5.4:

ArchiveDataLogs

False

If enabled, will move a message data log to the archive directory instead of deleting it.

DirectoryArchive

Null

Define the directory to move data logs to when they all the messages they contain have been consumed.

DatabaseLockedWaitDelay

10000

Time (ms) before trying to get acquire a the database lock (used by shared master/slave)

MaxAsyncJobs

10000

The maximum number of asynchronous messages that will be queued awaiting storage (should be the same as the number of concurrent MessageProducers)

ConcurrentStoreAndDispatchTopics

False

Enable the dispatching of Topic messages to interested clients to happen concurrently with message storage

ConcurrentStoreAndDispatchQueues

True

Enable the dispatching of Queue messages to interested clients to happen concurrently with message storage

2) AMQ persistent configuration

Property name

Default value

Comments

Directory

Activemq-data

The path to the directory to use to store the message store data and log files

UseNIO

True

Use NIO to write messages to the data logs

SyncOnWrite

False

Sync every write to disk

MaxFileLength

32mb

A hint to set the maximum size of the message data logs

PersistentIndex

True

Use a persistent index for the message logs. If this is false, an in-memory structure is maintained

MaxCheckpointMessageAddSize

4kb

The maximum number of messages to keep in a transaction before automatically committing

CleanupInterval

30000

Time (ms) before checking for a discarding/moving message data logs that are no longer used

IndexBinSize

1024

Default number of bins used by the index. The bigger the bin size-the better the relative performance of the index

IndexKeySize

ninety-six

The size of the index key-the key is the message id

IndexPageSize

16kb

The size of the index page-the bigger the page-the better the write performance of the index

DirectoryArchive

Archive

The path to the directory to use to store discarded data logs

ArchiveDataLogs

False

If true data logs are moved to the archive directory instead of being deleted

3) systemUsage configuration

Property name

Default value

Comments

MemoryUsage

20M

Amq uses memory size, which, according to the amq forum, should be greater than all durable desitination settings

The sum of memoryUsage, otherwise it will result in hard disk swap, affecting performance.

StoreUsage

1G

Kaha data storage size. If the setting is not enough, the performance will drop to 1 hair.

TempUsage

100M

Non-persistent messages are stored in the temp area

4) other configurations

Http://activemq.apache.org/nms/configuring.html

4.1) Failover Transport Options

Option Name

Default

Description

Transport.timeout

-1

Time that a send operation blocks before failing.

Transport.initialReconnectDelay

ten

Time in Milliseconds that the transport waits before attempting to reconnect the first time.

Transport.maxReconnectDelay

30000

The max time in Milliseconds that the transport will wait before attempting to reconnect.

Transport.backOffMultiplier

two

The amount by which the reconnect delay will be multiplied by if useExponentialBackOff is enabled.

Transport.useExponentialBackOff

True

Should the delay between connection attempt grow on each try up to the max reconnect delay.

Transport.randomize

True

Should the Uri to connect to be chosen at random from the list of available Uris.

Transport.maxReconnectAttempts

0

Maximum number of time the transport will attempt to reconnect before failing (0 means infinite retries)

Transport.startupMaxReconnectAttempts

0

Maximum number of time the transport will attempt to reconnect before failing when there has never been a connection made. (0 means infinite retries) (included in NMS.ActiveMQ v1.5.0+)

Transport.reconnectDelay

ten

The delay in milliseconds that the transport waits before attempting a reconnection.

Transport.backup

False

Should the Failover transport maintain hot backups.

Transport.backupPoolSize

one

If enabled, how many hot backup connections are made.

Transport.trackMessages

False

Keep a cache of in-flight messages that will flushed to a broker on reconnect

Transport.maxCacheSize

two hundred and fifty six

Number of messages that are cached if trackMessages is enabled.

Transport.updateURIsSupported

True

Update the list of known brokers based on BrokerInfo messages sent to the client.

4.2) Connection Options

Option Name

Default

Description

Connection.AsyncSend

False

Are message sent Asynchronously.

Connection.AsyncClose

True

Should the close command be sent Asynchronously

Connection.AlwaysSyncSend

False

Causes all messages a Producer sends to be sent Asynchronously.

Connection.CopyMessageOnSend

True

Copies the Message objects a Producer sends so that the client can reuse Message objects without affecting an in-flight message.

Connection.ProducerWindowSize

0

The ProducerWindowSize is the maximum number of bytes in memory that a producer will transmit to a broker before waiting for acknowledgement messages from the broker that it has accepted the previously sent messages. In other words, this how you configure the producer flow control window that is used for async sends where the client is responsible for managing memory usage. The default value of 0 means no flow control at the client. See also Producer Flow Control

Connection.useCompression

False

Should message bodies be compressed before being sent.

Connection.sendAcksAsync

False

Should message acks be sent asynchronously

Connection.messagePrioritySupported

True

Should messages be delivered to the client based on the value of the Message Priority header.

Connection.dispatchAsync

False

Should the broker dispatch messages asynchronously to the connection's consumers.

4.3) OpenWire Options

Option Name

Default

Description

WireFormat.stackTraceEnabled

False

Should the stack trace of exception that occur on the broker be sent to the client? Only used by openwire protocol.

WireFormat.cacheEnabled

False

Should commonly repeated values be cached so that less marshalling occurs? Only used by openwire protocol.

WireFormat.tcpNoDelayEnabled

False

Does not affect the wire format, but provides a hint to the peer that TCP nodelay should be enabled on the communications Socket. Only used by openwire protocol.

WireFormat.sizePrefixDisabled

False

Should serialized messages include a payload length prefix? Only used by openwire protocol.

WireFormat.tightEncodingEnabled

False

Should wire size be optimized over CPU usage? Only used by the openwire protocol.

WireFormat.maxInactivityDuration

30000

The maximum inactivity duration (before which the socket is considered dead) in milliseconds. On some platforms it can take a long time for a socket to appear to die, so we allow the broker to kill connections if they are inactive for a period of time. Use by some transports to enable a keep alive heart beat feature. Set to a value

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.