Draft Forbes Group Website (Build by Nikola). The official site is hosted at:
License: GPL3
ubuntu2004
Using Nikola to blog with ipython notebooks.
http://www.damian.oquanta.info/posts/ipython-plugin-for-nikola-updated.html
Installing Nikola
Update January 2020: I recommend installing Nikola from a conda environment.yml
file. I keep one on anaconda cloud so you can just go:
There is no conda recipe yet for Nikola, so it needs to be pip installed.
To uses some html features like SASS, I needed to install npm
which I did at the system level.
Old Notes and Problems
I had some trouble pip-installing nikola with pip so had to install a few of the dependencies first with conda.
Fetching package metadata: ..........
Solving package specifications: .
# All requested packages already installed.
# packages in environment at /data/apps/anaconda/1.3.1:
#
pillow 2.7.0 py27_0
Requirement already satisfied (use --upgrade to upgrade): nikola in /data/src/python/git/nikola
Requirement already satisfied (use --upgrade to upgrade): livereload in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages
Collecting webassets
Downloading webassets-0.10.1.tar.gz (167kB)
100% |################################| 167kB 386kB/s
Requirement already satisfied (use --upgrade to upgrade): doit>=0.23.0 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): Pygments>=1.6 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): Pillow>=2.4.0 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): python-dateutil==2.4.0 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): docutils>=0.12 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): mako>=1.0.0 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): unidecode>=0.04.16 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): lxml>=3.3.5 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): yapsy>=1.10.423 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): PyRSS2Gen>=1.1 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): logbook>=0.7.0 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola)
Collecting setuptools>=5.4.1 (from nikola)
Using cached setuptools-12.1-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): natsort>=3.5.2 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola)
Requirement already satisfied (use --upgrade to upgrade): tornado in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from livereload)
Requirement already satisfied (use --upgrade to upgrade): six in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from doit>=0.23.0->nikola)
Requirement already satisfied (use --upgrade to upgrade): macfsevents in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from doit>=0.23.0->nikola)
Requirement already satisfied (use --upgrade to upgrade): MarkupSafe>=0.9.2 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from mako>=1.0.0->nikola)
Installing collected packages: setuptools, webassets
Found existing installation: distribute 0.6.45
Uninstalling distribute-0.6.45:
Successfully uninstalled distribute-0.6.45
Running setup.py install for webassets
Installing webassets script to /data/apps/anaconda/1.3.1/bin
Successfully installed setuptools-0.6rc11 webassets-0.10.1
Fetching package metadata: ..........
# All requested packages already installed.
# packages in environment at /data/apps/anaconda/1.3.1:
#
lxml 3.4.2 py27_0
Compiler
I had trouble installing macfsevents
using the HPC compilers, so I had to disable them. When using the Xcode compilers, everything worked.
Lack of Root Permissions
If you don't have root access, then running npm
or gem
might cause issues (i.e. on Sage Mathcloud.) For npm
, the suggestion here to:
works.
New Blog/Website
These new instructions are based on my custom themes. See below to base your blog on public themes.
Install Nikola (above) and activate the
blog
environment.You can also use
anaconda-client
to do this from myenvironment.blog.yml
file on anaconda cloud. Note: Make sure there is no localenvironment.yml
file, or creating the environment from anaconda cloud may fail.
CoCalc
Here is a complete walk-through using CoCalc. The example will be for the iSciMath main website. We do a couple of things here:
Store all images in a separate repo since these can be big.
Here is the play by play:
Initial Setup
Create a new project on CoCalc and enable internet access. (This will require that you have some access to at least one upgrade... they needed to do this to prevent people from abusing the resources.)
Create and activate the
blog
environment.As a check, you should now be able to run Nikola:
Create a Jupyter kernel (optional). Here we create a Jupyter Kernel so that we can activate the
blog
kernel fromREADME.ipynb
providing some interactive instructions. This is not needed for the general functionality of Nikola.Here is how I modified the
kernel.json
file:Create a repo for your website and add some files:
Use Nikola to create the site.
The main thing this does is create the
conf.py
file, which we should now commit to version control.
(blog) ~/repositories/iscimath_website hg com -m "Initial commit of Nikola-generated conf.py" ```
Content
Now we add some content. For demo purposes, we add simple markdown content from geometricanalysis.org.
First we create a repo for the assets, and download some images
Link this into our main repo
Now create an index file. We can do this in a variety of formats, but ReStructuredText gives us more control than Markdown.
At this point we now need to edit the conf.py
file to convert this to a website rather than a blog. We follow the process described here. The following was changed:
Commit these changes:
Preview
Now you can build and view the page:
This will be hosted on CoCalc as described here as long as the nikola server is running. Only users authenticated to the CoCalc project can see it.
I have also put these commands into the Makefile, so one can do
Theming
To use my new themes, based on the zen-jinja theme, I needed to install that SASS plugin:
and add the following to the conf.py
file:
Setup Blog (Old Version)
nikola init test_blog
Answer the questions
Follow the instructions on http://www.damian.oquanta.info/posts/ipython-plugin-for-nikola-updated.html for setting up your blog to work with IPython notebooks:
nikola install_theme zen-ipython
Add the following to your
conf.py
file:
Generating Metadata
I have a bunch of notebooks I want to "blog" but need to generate metadata. Here is some code to do this programatically from the file information.
Updating Nikola
If you need the latest development version of Nikola, you can install form sources:
I needed this before v7.3.1 to fix issue #1582.
Later I needed this before version v7.8.0 to deal with Ipython version 4.0 deprecations.
Later I needed this before this commit when sys.path
was not including the root directory (where I keep a module that needs to be imported.)
MathJax
If you want to use LaTeX in your notebook, be sure to add the following to your conf.py
file.
Comments
I am trying the Disqus comment system since this was recommended in the Nikola manual. I had to create a Disqus account, and then register a new site to get the shortname
. Some immediately apparent issues:
Disqus puts a lot of things in a global namespace. For example, the username (my usual handle was already taken) and when registering a new site, you need to guess a unique identifier
<something>.disqus.com
.
Website
Nikola can also be used for a static website. Here is the list of packages and versions I used when doing this:
# packages in environment at /data/apps/conda/envs/blog:
#
# Name Version Build Channel
aiohttp 3.6.2 py36h1de35cc_0
appnope 0.1.0 py36hf537a9a_0
argh 0.26.2 py36_0
argon2-cffi 20.1.0 py36haf1e3a3_1
async-timeout 3.0.1 py36_0
async_generator 1.10 py36h28b3542_0
attrs 20.2.0 py_0
babel 2.8.0 py_0
backcall 0.2.0 py_0
bleach 3.2.1 py_0
blinker 1.4 py36_0
brotlipy 0.7.0 py36haf1e3a3_1000
ca-certificates 2020.7.22 0
certifi 2020.6.20 py36_0
cffi 1.14.3 py36hed5b41f_0
chardet 3.0.4 py36_1003
cloudpickle 1.6.0 py_0
cryptography 3.1.1 py36hddc9c9b_0
decorator 4.4.2 py_0
defusedxml 0.6.0 py_0
docutils 0.16 py36_1
doit 0.32.0 py36_0
entrypoints 0.3 py36_0
freetype 2.10.2 ha233b18_0
ghp-import2 1.0.1 pypi_0 pypi
icu 58.2 h0a44026_3
idna 2.10 py_0
idna_ssl 1.1.0 py36_0
importlib-metadata 1.7.0 py36_0
importlib_metadata 1.7.0 0
ipykernel 5.3.4 py36h5ca1d4c_0
ipython 7.16.1 py36h5ca1d4c_0
ipython_genutils 0.2.0 py36_0
jedi 0.17.2 py36_0
jinja2 2.11.2 py_0
jpeg 9b he5867d9_2
jsonschema 3.2.0 py36_1
jupyter_client 6.1.7 py_0
jupyter_core 4.6.3 py36_0
jupyterlab_pygments 0.1.2 py_0
lcms2 2.11 h92f6f08_0
libcxx 10.0.0 1
libedit 3.1.20191231 h1de35cc_1
libffi 3.3 hb1e8313_2
libiconv 1.16 h1de35cc_0
libpng 1.6.37 ha441bb4_0
libsodium 1.0.18 h1de35cc_0
libtiff 4.1.0 hcb84e12_1
libxml2 2.9.10 h3b9e6c8_1
libxslt 1.1.34 h83b36ba_0
lxml 4.5.2 py36h63b7cb6_0
lz4-c 1.9.2 hb1e8313_1
macfsevents 0.8.1 py36h1de35cc_0
mako 1.1.3 py_0
markdown 3.3 py36_0
markupsafe 1.1.1 py36h1de35cc_0
mistune 0.8.4 py36h1de35cc_0
mmf_setup 0.3.0 py_0 mforbes
multidict 4.7.6 py36haf1e3a3_1
natsort 7.0.1 py_0
nbclient 0.5.0 py_0
nbconvert 6.0.7 py36_0
nbformat 5.0.7 py_0
nbstripout 0.3.9 pyh9f0ad1d_0 conda-forge
ncurses 6.2 h0a44026_1
nest-asyncio 1.4.1 py_0
nikola 8.1.1 pypi_0 pypi
notebook 6.1.4 py36_0
olefile 0.46 py36_0
openssl 1.1.1h haf1e3a3_0
packaging 20.4 py_0
pandoc 2.10.1 0
pandocfilters 1.4.2 py36_1
parso 0.7.0 py_0
pathtools 0.1.2 py_1
pexpect 4.8.0 py36_0
pickleshare 0.7.5 py36_0
piexif 1.1.3 py_2 conda-forge
pillow 7.2.0 py36ha54b6ba_0
pip 20.2.3 py36_0
pipdeptree 1.0.0 pypi_0 pypi
prometheus_client 0.8.0 py_0
prompt-toolkit 3.0.7 py_0
ptyprocess 0.6.0 py36_0
pycparser 2.20 py_2
pygments 2.7.1 py_0
pyopenssl 19.1.0 py_1
pyparsing 2.4.7 py_0
pyrsistent 0.17.3 py36haf1e3a3_0
pyrss2gen 1.1 pypi_0 pypi
pysocks 1.7.1 py36_0
python 3.6.12 h26836e1_2
python-dateutil 2.8.1 py_0
python-hglib 2.6.1 py_0 mforbes
pytz 2020.1 py_0
pyyaml 5.3.1 py36haf1e3a3_1
pyzmq 19.0.2 py36hb1e8313_1
readline 8.0 h1de35cc_0
requests 2.24.0 py_0
send2trash 1.5.0 py36_0
setuptools 50.3.0 py36h0dc7051_1
six 1.15.0 py_0
smartypants 2.0.0 pyh9f0ad1d_2 conda-forge
sqlite 3.33.0 hffcf06c_0
terminado 0.8.3 py36_0
testpath 0.4.4 py_0
tk 8.6.10 hb0a8c7a_0
tornado 6.0.4 py36h1de35cc_1
traitlets 4.3.3 py36_0
typing_extensions 3.7.4.3 py_0
typogrify 2.0.7 py36_0 conda-forge
unidecode 1.1.1 py_0
urllib3 1.25.10 py_0
watchdog 0.10.3 py36haf1e3a3_0
wcwidth 0.2.5 py_0
webencodings 0.5.1 py36_1
wheel 0.35.1 py_0
xz 5.2.5 h1de35cc_0
yaml 0.2.5 haf1e3a3_0
yapsy 1.12.2 py_0 conda-forge
yarl 1.6.0 py36haf1e3a3_0
zeromq 4.3.2 hb1e8313_3
zipp 3.3.0 py_0
zlib 1.2.11 h1de35cc_3
zstd 1.4.5 h41d2c2f_0
Now version control the generated files:
We now switch the theme to zen-ipython
which requires installing some themes:
Organization
In addition to static content, I would like to have posts on my webpage describe research, projects, etc. These can be organized in "sections" by placing them in appropriate directories in posts
and then specifying the output directories in the POSTS
variable in conf.py
. One can then filter them, for example, with {% for post in posts if post.section_name().lower() == "highlights" %}
in a template.
I use this with a special template templates/research_highlights_list.tmpl
to format the research highlights on my index page.
Customization
Themes
Here we collect some notes about how themes work and can be customized. Themes are stored in the themes
directory. For example, themes/zen-ipython
. I have a separate repo for custom themes which I include in _ext/forbes_group_website_theme
and then link into the themes
.
A Nikola site can be customized in several ways:
conf.py
: Many customizations appear in this file, including some that might not be obvious:INDEX_READ_MORE_LINK
: This variable is used to customize the "Read more" link. We used this to modify these to look like a button matching the "Editorial" style.NAVIGATION_LINKS
: These appear in the side-bar, so you can specify the icons here (from Font Awesome if you use thezen-*
themes).POSTS
/PAGES
: Here you can specify which template files are used for posts and pages.
Modify the templates. To do this, I found it easiest to copy all of the inherited template files into a new theme's
template
directory so I can see what is happening. Here are some important templates:index.tmpl
: Used for the index page collecting your blog posts.post.tmpl
: Used for the individual blog posts. This can be selected inPOSTS
variable ofconf.py
. These can also be controlled in the actual posts with thetemplate
variable.story.tmpl
: Used for static pages. This can be selected inPAGES
variable ofconf.py
. These can also be controlled in the actual posts with thetemplate
variable.
zen-ipython
This directory contains a file themes/zen-ipython/parent
that specify the parent theme - in this case themes/zen-jinja
. Thus, to find assets, etc. one might need to look in several places. Here is the full hierarchy for the themes/zen-ipython
theme:
themes/base-jinja
(locate files withnikola theme -g base-jinja
)themes/base
(locate files withnikola theme -g base
)
The last two are pre-installed themes whose location on disk can be found using the specified command. I copied them into my site folder for easy reference.
Icons
The zen-ipython
theme uses icons from Font Awesome. To see the available icons and names, look at the icon list for the appropriate version (version 3.2.1 here). You can then specify the appropriate fonts in conf.py
.
Less/Sass
The zen themes use [less
], which they tell you in the Readme.md
file can be enabled by setting USE_BUNDLES = False
in conf.py
after installing the lessc
compiler and nikola plugin -i less
, however, you also need to rename/remove themes/zen-jinja/assets/css/main.cssmain.css
file or else you will get the following error (since it is also built with lessc
...)
Targets for less
are specified in the file themes/zen-jinja/less/targets.
I am using a custom theme which provides sass
stylesheets. This requires installing the sass
compiler and then one can use the sass
plugin to generate appropriate CSS files. This theme was inheriting from the zen-jinja
theme and so the main.css
file was clashing with the previous file specified in themes/zen-jinja/less/targets. To resolve this, I added a custom less directory in the new theme with an empty themes/html5up-editorial/less/targets
file.
Jinja2
Here are some tips for working with Jinja2 (see the references section below for details.)
You can see the value of variables using code like this:
{{ post.tags|pprint }}
or if you need to{{ post.__dict__.keys()|pprint }}
.
Images
One issue I had is how to include images in posts. The usual approach for notebooks is to include the images in a file relative to the notebook. For example, if the files are in a folder images
then one can refer to this as
There are two issues with this approach:
Nikola will not copy these images to the output. This can be overcome by storing the files in the
images
folder at the root of your project, and then specifying where these should appear in the output by specifyingIMAGE_FOLDERS = {'images': 'images'}
(the default value) in yourconf.py
file. You can then symlink this file to your posts directory.The second issue is that this image will produce a link like this in the output:
I.e. this is a link relative to the post which would require a complicated set of non-standard
IMAGE_FOLDERS = {'images': 'images'}
mappings.
The standard Nikola solution is to place the images in a images/
folder in the top level of your project as above with IMAGE_FOLDERS = {'images': 'images'}
in your conf.py
file. Then refer with an absolute address:
Unfortunately, this breaks in the notebook rendering which cannot see higher-level directories. For example, if you start the Jupyter notebook server in the same directory as the post, then one would have to use the following absolute path in the notebook in order to see the image:
This location could then be hacked with Nikola using IMAGE_FOLDERS = {'images': 'tree/images'}
.
Solutions
I see three possible solutions:
Use the modified "tree" hack above:
Place images in
Symlink this to an images folder next to your posts.
Refer to your images as
Use the following mapping in your
conf.py
file:
Mirror the structure of your posts in your images directory. For example, if your posts are in
posts/highlights/my_post.ipynb
, then:Place images in
Symlink these images to your posts directory:
Refer to images with a relative path:
Map the images to the root level so they are included in the relative path:
Use the Nikola recommend approach of linking to an absolute path and extend the jupyter notebook server to properly redirect these requests.
Place images in
Refer to your images using an absolute path
Run
jupyter notebook
from the top level (whereimages
is) after installing and loading my notebook server extension mmf_nikola_nbserver_extension (this is done in the filejupyter_notebook_config.py
:
I am presently using the last approach.
Notebook Conversion
The notebooks are converted to HTML through the nbcovert
process. This can be customized through templates or providing plugins.
The
'IPYNB_CONFIG'
variable inconf.py
is passed to theHTMLExporter
. To see possible options, runjupyter nbconvert --help-all
. For example, I have added a preprocessormmf_nikola_nbserver_extension.ArXivLinks
which replaces references to preprints with a URL. This is enabled as follows inconf.py
:IPYNB_CONFIG = { 'Exporter': { 'preprocessors': ['mmf_nikola_nbserver_extension.ArXivLinks'] } }
Plugins
Sometimes you just can't seem to do what you want with the existing tools. One way of extending Nikola is to use a plugin. First check the Nikola Plugins to see if you can find something, otherwise you can create your own in the plugins
directory. To add a simple reStructuredText directive, for example, you can install and copy the book_figure plugin, which is a fairly simple plugin that creates a book_figure
directive:
then edit the plugin.
There are also some useful plugins: here is a list of the plugins I sometimes use:
References
People who have customized Nikola and describe the process:
How I customized my Nikola-powered site: A nice discussion about customizing a website powered with Nikola. Based on the
ipython
theme.
Official Documentation:
Nikola Internals Describes the type of data available for use in templates etc.
Jinja Templating:
Some nice HTML and CSS templates:
HTML5 UP: Some nice, responsive, and clean HTML and CSS templates. We have tried to massage one of these into a website.