Related to jupyter installation

1. Install Python 3.5.7

wget https://www.python.org/ftp/python/3.5.7/Python-3.5.7.tgz

tar -zxvf Python-3.5.7.tgz

cd Python-3.5.7

./ configure --prefix=/usr/local/python-3.5.7

make

make install

ln /usr/local/python-3.5.7/bin/python3 -s /usr/bin/python3

ln /usr/local/python-3.5.7/bin/pip3 -s /usr/bin/pip3

2. Install jupyter

See www.cnblogs.com/noahzhixiao/p/9985566.html

Install ipython, jupyter

pip install ipython

pip install jupyter

generate a configuration file

[root@50eb5057baac /]# jupyter notebook --generate-config

Writing default config to: /root/.jupyter/jupyter_notebook_config.py

generate a password

root@50eb5057baac /]# ipython Python 3.5.1 (default, Oct 21 2016, 21:37:19)

Type 'copyright', 'credits' or 'license' for more information IPython 6.2.1 -- An enhanced Interactive Python. Type '? ' for help.

In [1]: from notebook.auth import passwd

In [2]: passwd()

Enter password:

Verify password:

Out[2]: 'sha1:43b95b731276:5d330ee6f6054613b3ab4cc59c5048ff7c70f549' In [3]:

Modify default profiles

vi /root/.jupyter/jupyter_notebook_config.py

c.NotebookApp.ip='' #Set ip to access notebook, indicating all IPs, here set ip to all accessible c.NotebookApp.password = u'sha1:5df252f58b7f:bf65d53125bb36c085162b3780377f66d73972d1' #Fill in the ciphertext just generated c.NotebookApp.open_browser = False #Prevent notebook from automatically opening the browser when it starts (in linux servers, it is generally ssh command line access, there is no graphical interface. So it doesn't help to start.)

NotebookApp.port =8889 #Specifies the port to access, default is 8888.

And remove the previous comment #

Start jupyter notebook --no-browser

[plain] view plain copy [root@346086094cbe /]# jupyter notebook --allow-root [W 17:17:04.106 NotebookApp] WARNING: The notebook server is listening on all IP addresses and not using encryption. This is not recommended. [I 17:17:04.111 NotebookApp] Serving notebooks from local directory: / [I 17:17:04.112 NotebookApp] 0 active kernels [I 17:17:04.112 NotebookApp] The Jupyter Notebook is running at: [I 17:17:04.112 NotebookApp] http://[all ip addresses on your system]:8889/ [I 17:17:04.112 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).

Then you can type your address http://yourip:8889/into your browser and see the following interface.

installation is successful

Start Jupyter's development window and click new3. Install dependency packages in the upper right corner.

Under normal circumstances, you can't open jupyter, so you need to install dependency packages.

See www.cnblogs.com/ligh-test/p/10028850.html

no mudole named _sqlite3mportError: dynamic module does not define module export function (PyInit__caffe)

In fact, these two problems are because python3 and above versions do not support sqlite3, the following method is pro-test no problem, from (sparkexpert god)

(1) Package for installing sqlite3

$ wget https://www.sqlite.org/2017/sqlite-autoconf-3170000.tar.gz --no-check-certificate

$ tar zxvf sqlite-autoconf-3170000.tar.gz

$ cd sqlite-autoconf-3170000

$ ./ configure --prefix=/usr/local/sqlite3 --disable-static --enable-fts5 --enable-json1 CFLAGS="-g -O2 -DSQLITE_ENABLE_FTS3=1 -DSQLITE_ENABLE_FTS4=1 -DSQLITE_ENABLE_RTREE=1"

(2) Recompile Python 3

$ cd Python 3.5.7 (Python download unzipped directory)

$ LD_RUN_PATH=/usr/local/sqlite3/lib ./ configure LDFLAGS="-L/usr/local/sqlite3/lib" CPPFLAGS="-I /usr/local/sqlite3/include"

$ LD_RUN_PATH=/usr/local/sqlite3/lib make

$ LD_RUN_PATH=/usr/local/sqlite3/lib sudo make install

After the above steps, there should be no problem. Enter python3 in the console to enter the environment.

import sqlite3 did not report an error ok

4. Add spark dependency

Modify.bashrc

Add:

export SPARK_HOME=/opt/cloudera/parcels/CDH/lib/spark

export PATH=$SPARK_HOME/bin:$PATH

export PYTHONPATH=/usr/local/python-3.5.7:$PYTHONPATH

export PYSPARK_PYTHON=python3