Book a Demo!
CoCalc Logo Icon
StoreFeaturesDocsShareSupportNewsAboutPoliciesSign UpSign In
Download

Draft Forbes Group Website (Build by Nikola). The official site is hosted at:

https://labs.wsu.edu/forbes

5913 views
License: GPL3
ubuntu2004
Kernel: Python [conda env:blog]

Installing Nikola

Update January 2020: I recommend installing Nikola from a conda environment.yml file. I keep one on anaconda cloud so you can just go:

conda env create mforbes/blog such as the following: ```yml # environment.yml name: blog channels: - defaults - conda-forge - mforbes dependencies: - jinja2 # Jinja2 Template engine for my styles. - nbstripout # Cleaning output from notebooks - typogrify # Required for the typogrify filter # These are needed by Nikola. We install them so Conda has control. - babel>=2.6.0 - blinker>=1.3 - docutils>=0.13 - doit>=0.32.0 - lxml>=3.3.5 - mako>=1.0.0 - markdown>=3.0.0 - natsort>=3.5.2 - piexif>=1.0.3 - pillow>=2.4.0 - pygments>=1.6 - python-dateutil>=2.6.0 - requests>=2.2.0 - setuptools>=24.2.0 - unidecode>=0.04.16 - yapsy>=1.11.223 # Additional dependencies not in conda - they will be installed by pip. #- PyRSS2Gen>=1.1 # Needed of ipynb format - nbconvert - notebook>=4.0.0 # Needed for nikola auto - watchdog>=0.8.3 # Conda version gives "Failed to import fsevents. Fall back to kqueue" - aiohttp # My packages. - mmf_setup - pip - pip: #- nikola>=8.1.1 # Need dev version for this issue with templates # https://github.com/getnikola/nikola/issues/3457 - git+https://github.com/getnikola/nikola.git@master - pipdeptree #- watchdog>=0.8.3 - ghp-import2 # Needed for github_deploy #- pyrss2gen>=1.1 #- python-hglib==2.6.1

There is no conda recipe yet for Nikola, so it needs to be pip installed.

To uses some html features like SASS, I needed to install npm which I did at the system level.

conda env create -f environment.blog.yml conda activate blog npm install -g less # lessc compiler required for CSS in zen- themes sudo gem install sass # sass compiler required for SASS based CSS nikola plugin -i less nikola plugin -i sass

Old Notes and Problems

I had some trouble pip-installing nikola with pip so had to install a few of the dependencies first with conda.

# This worked with python 3 in an anaconda install. !pip install nikola webassets !conda install ws4py watchdog # Required for nikola auto !pip install typogrify # Required for the typogrify filter !npm install -g less # lessc compiler required for CSS in zen- themes !sudo gem install sass # sass compiler required for SASS based CSS !nikola plugin -i less !nikola plugin -i sass
Collecting nikola Using cached Nikola-7.7.6-py2.py3-none-any.whl Requirement already satisfied (use --upgrade to upgrade): webassets in /data/apps/anaconda/envs/work/lib/python2.7/site-packages Requirement already satisfied (use --upgrade to upgrade): natsort>=3.5.2 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola) Requirement already satisfied (use --upgrade to upgrade): docutils>=0.12 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola) Requirement already satisfied (use --upgrade to upgrade): doit<=0.29.0,>=0.28.0 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola) Requirement already satisfied (use --upgrade to upgrade): unidecode>=0.04.16 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola) Requirement already satisfied (use --upgrade to upgrade): python-dateutil>=2.4.0 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola) Requirement already satisfied (use --upgrade to upgrade): requests>=2.2.0 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola) Requirement already satisfied (use --upgrade to upgrade): Pygments>=1.6 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola) Requirement already satisfied (use --upgrade to upgrade): lxml>=3.3.5 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola) Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola) Requirement already satisfied (use --upgrade to upgrade): husl>=4.0.2 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola) Requirement already satisfied (use --upgrade to upgrade): logbook>=0.7.0 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola) Requirement already satisfied (use --upgrade to upgrade): Pillow>=2.4.0 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola) Requirement already satisfied (use --upgrade to upgrade): PyRSS2Gen>=1.1 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola) Requirement already satisfied (use --upgrade to upgrade): mako>=1.0.0 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola) Requirement already satisfied (use --upgrade to upgrade): Yapsy>=1.11.223 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from nikola) Requirement already satisfied (use --upgrade to upgrade): setuptools>=5.4.1 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages/setuptools-20.1.1-py2.7.egg (from nikola) Requirement already satisfied (use --upgrade to upgrade): six in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from doit<=0.29.0,>=0.28.0->nikola) Requirement already satisfied (use --upgrade to upgrade): macfsevents in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from doit<=0.29.0,>=0.28.0->nikola) Requirement already satisfied (use --upgrade to upgrade): configparser in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from doit<=0.29.0,>=0.28.0->nikola) Requirement already satisfied (use --upgrade to upgrade): MarkupSafe>=0.9.2 in /data/apps/anaconda/envs/work/lib/python2.7/site-packages (from mako>=1.0.0->nikola) Installing collected packages: nikola Successfully installed nikola-7.7.6
# I needed this before with python 2 !conda install pillow !pip install nikola livereload webassets !conda update lxml
Fetching package metadata: .......... Solving package specifications: . # All requested packages already installed. # packages in environment at /data/apps/anaconda/1.3.1: # pillow 2.7.0 py27_0 Requirement already satisfied (use --upgrade to upgrade): nikola in /data/src/python/git/nikola Requirement already satisfied (use --upgrade to upgrade): livereload in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages Collecting webassets Downloading webassets-0.10.1.tar.gz (167kB) 100% |################################| 167kB 386kB/s Requirement already satisfied (use --upgrade to upgrade): doit>=0.23.0 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola) Requirement already satisfied (use --upgrade to upgrade): Pygments>=1.6 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola) Requirement already satisfied (use --upgrade to upgrade): Pillow>=2.4.0 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola) Requirement already satisfied (use --upgrade to upgrade): python-dateutil==2.4.0 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola) Requirement already satisfied (use --upgrade to upgrade): docutils>=0.12 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola) Requirement already satisfied (use --upgrade to upgrade): mako>=1.0.0 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola) Requirement already satisfied (use --upgrade to upgrade): unidecode>=0.04.16 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola) Requirement already satisfied (use --upgrade to upgrade): lxml>=3.3.5 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola) Requirement already satisfied (use --upgrade to upgrade): yapsy>=1.10.423 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola) Requirement already satisfied (use --upgrade to upgrade): PyRSS2Gen>=1.1 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola) Requirement already satisfied (use --upgrade to upgrade): logbook>=0.7.0 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola) Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola) Collecting setuptools>=5.4.1 (from nikola) Using cached setuptools-12.1-py2.py3-none-any.whl Requirement already satisfied (use --upgrade to upgrade): natsort>=3.5.2 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from nikola) Requirement already satisfied (use --upgrade to upgrade): tornado in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from livereload) Requirement already satisfied (use --upgrade to upgrade): six in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from doit>=0.23.0->nikola) Requirement already satisfied (use --upgrade to upgrade): macfsevents in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from doit>=0.23.0->nikola) Requirement already satisfied (use --upgrade to upgrade): MarkupSafe>=0.9.2 in /data/apps/anaconda/1.3.1/lib/python2.7/site-packages (from mako>=1.0.0->nikola) Installing collected packages: setuptools, webassets Found existing installation: distribute 0.6.45 Uninstalling distribute-0.6.45: Successfully uninstalled distribute-0.6.45 Running setup.py install for webassets Installing webassets script to /data/apps/anaconda/1.3.1/bin Successfully installed setuptools-0.6rc11 webassets-0.10.1 Fetching package metadata: .......... # All requested packages already installed. # packages in environment at /data/apps/anaconda/1.3.1: # lxml 3.4.2 py27_0

Compiler

I had trouble installing macfsevents using the HPC compilers, so I had to disable them. When using the Xcode compilers, everything worked.

Lack of Root Permissions

If you don't have root access, then running npm or gem might cause issues (i.e. on Sage Mathcloud.) For npm, the suggestion here to:

npm config set prefix ~/.npm cat > ~/.bash_aliases <<EOF # Add local npm package to path # http://stackoverflow.com/a/23889603/1088938 # http://stackoverflow.com/a/40830016/1088938 export PATH="$PATH:$HOME/.npm/bin" export GEM_HOME="$HOME/.gem" export PATH="$PATH:$GEM_HOME/bin" EOF
. ~/.bash_aliases npm install less gem install sass

works.

New Blog/Website

These new instructions are based on my custom themes. See below to base your blog on public themes.

  1. Install Nikola (above) and activate the blog environment.

    conda env create -f environment.blog.yml

    You can also use anaconda-client to do this from my environment.blog.yml file on anaconda cloud. Note: Make sure there is no local environment.yml file, or creating the environment from anaconda cloud may fail.

    conda install anaconda-client conda env create mforbes/blog

CoCalc

Here is a complete walk-through using CoCalc. The example will be for the iSciMath main website. We do a couple of things here:

  • Store all images in a separate repo since these can be big.

Here is the play by play:

Initial Setup

  1. Create a new project on CoCalc and enable internet access. (This will require that you have some access to at least one upgrade... they needed to do this to prevent people from abusing the resources.)

  2. Create and activate the blog environment.

    anaconda2020 # CoCalc-specific way of activating conda. conda install anaconda-client # Not needed on CoCalc, but may be needed elsewhere conda env create mforbes/blog # Make sure there is no local environment.yml file! conda clean -y --all # Free disk space from downloads. conda activate blog

    As a check, you should now be able to run Nikola:

    $ nikola --version Nikola v8.1.1
  3. Create a Jupyter kernel (optional). Here we create a Jupyter Kernel so that we can activate the blog kernel from README.ipynb providing some interactive instructions. This is not needed for the general functionality of Nikola.

    mkdir -p ~/.local/share/jupyter/kernels/ cp -r /ext/anaconda2020.02/share/jupyter/kernels/python3 ~/.local/share/jupyter/kernels/blog-py vi ~/.local/share/jupyter/kernels/blog-py/kernel.json

    Here is how I modified the kernel.json file:

    #kernel.json { "argv": [ "/home/user/.conda/envs/blog/bin/python", "-m", "ipykernel_launcher", "-f", "{connection_file}" ], "display_name": "Blog", "language": "python" }
  4. Create a repo for your website and add some files:

    mkdir -p ~/repositories/iscimath_website cd ~/repositories/iscimath_website hg init tee .hgignore <<EOF syntax: glob syntax: regexp ^.output/ ^.cache/ EOF tee Makefile <<EOF auto: echo "https://cocalc.com/ff1cb986-feb2-4122-b55d-1700a5fd8553/server/8000" nikola auto -a 0.0.0.0 -p 8000 .PHONY: auto EOF
  5. Use Nikola to create the site.

    (blog) ~/repositories/iscimath_website$ nikola init . Creating Nikola Site ==================== This is Nikola v8.1.1. We will now ask you a few easy questions about your new site. If you do not want to answer and want to go with the defaults instead, simply restart with the `-q` parameter. --- Questions about the site --- Site title [My Nikola Site]: iSciMath: Integrated Science and Mathematics Site author [Nikola Tesla]: Michael McNeil Forbes Site authors e-mail [n.tesla@example.com]: cdlg.iscimath+admin@gmail.com Site description [This is a demo site for Nikola.]: iSciMath Website. Site URL [https://example.com/]: https://www.iscimath.org/ Enable pretty URLs (/page/ instead of /page.html) that do not need web server configuration? [Y/n] --- Questions about languages and locales --- We will now ask you to provide the list of languages you want to use. Please list all the desired languages, comma-separated, using ISO 639-1 codes. The first language will be used as the default. Type '?' (a question mark, sans quotes) to list available languages. Language(s) to use [en]: Please choose the correct time zone for your blog. Nikola uses the tz database. You can find your time zone here: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones Time zone [Etc/UTC]: America/Los_Angeles Current time in America/Los_Angeles: 13:43:22 Use this time zone? [Y/n] --- Questions about comments --- You can configure comments now. Type '?' (a question mark, sans quotes) to list available comment systems. If you do not want any comments, just leave the field blank. Comment system: That is it, Nikola is now configured. Make sure to edit conf.py to your liking. If you are looking for themes and addons, check out https://themes.getnikola.com/ and https://plugins.getnikola.com/. Have fun! [2020-10-10 20:47:27] INFO: init: Created empty site at .. (blog) ~/repositories/iscimath_website$ tree . conf.py files galleries images listings pages posts 6 directories, 1 file

    The main thing this does is create the conf.py file, which we should now commit to version control.

(blog) ~/repositories/iscimath_websitehgaddadding.hgignoreaddingconf.py(blog) /repositories/iscimathwebsite hg add adding .hgignore adding conf.py (blog) ~/repositories/iscimath_website hg com -m "Initial commit of Nikola-generated conf.py" ```

Content

Now we add some content. For demo purposes, we add simple markdown content from geometricanalysis.org.

  1. First we create a repo for the assets, and download some images

    mkdir -p ~/repositories/iscimath_website_assets cd ~/repositories/iscimath_website_assets hg init mkdir images wget https://images.squarespace-cdn.com/content/v1/5cb91b1c70468025fd21e98b/1555698446161-7CZZ16Y5BMUKXK4BXY15/ke17ZwdGBToddI8pDm48kHtOJuqdoEFYKNzpMbCdjQ1Zw-zPPgdn4jUwVcJE1ZvWQUxwkmyExglNqGp0IvTJZamWLI2zvYWH8K3-s_4yszcp2ryTI0HqTOaaUohrI8PI84wy8WKf6jeaYut84yuMMxu-RozL28G4_z9WCTpl1xsKMshLAGzx4R3EDFOm1kBS/events.jpg -O images/banner1.jpg wget https://images.squarespace-cdn.com/content/v1/5cb91b1c70468025fd21e98b/1555728634489-WHSXTS0E32WU1OHYS8Y6/ke17ZwdGBToddI8pDm48kNaADO1I1V8iOHJQMUQI2H1Zw-zPPgdn4jUwVcJE1ZvWQUxwkmyExglNqGp0IvTJZamWLI2zvYWH8K3-s_4yszcp2ryTI0HqTOaaUohrI8PIBCpzMMEguc8VVUBIRTMwFWs2R71F1SeBH0ohOKrYXboKMshLAGzx4R3EDFOm1kBS/upcomingevents.jpg -O images/ideas.jpg hg add hg commit -m "IMG: Added images"
  2. Link this into our main repo

    cd ~/repositories/iscimath_website mkdir _ext ln -s ~/repositories/iscimath_website_assets _ext/ rmdir images ln -s _ext/iscimath_website_assets/images . hg add images
  3. Now create an index file. We can do this in a variety of formats, but ReStructuredText gives us more control than Markdown.

    .. title: Center for Geometric Analysis and Data .. slug: index .. date: 2020-10-10 15:03:35 UTC-07:00 .. tags: .. category: .. link: .. description: .. type: text .. image:: images/banner1.jpg :width: 100% :alt: Alternative text The Center for Geometric Analysis and Data is the research group led by Kevin R. Vixie. As the name suggests, the research is focused on topics at the intersection of analysis and geometry as well as those suggested by challenges from data science or data generated by scientific questions. To find out more about Kevin and his circle of collaborators, visit the Group Members page, peruse the notes and papers, read the blog, or follow the links that abound throughout. EOF

At this point we now need to edit the conf.py file to convert this to a website rather than a blog. We follow the process described here. The following was changed:

# Modified conf.py ... # Website: remove destination directory to generate pages in the root directory PAGES = ( ("pages/*.rst", "", "page.tmpl"), ("pages/*.md", "", "page.tmpl"), ("pages/*.txt", "", "page.tmpl"), ("pages/*.html", "", "page.tmpl"), ) ... # Website: we will provide our own index, so blog stuff goes here instead INDEX_PATH = "blog"

Commit these changes:

hg add hg com -m "Initial version of website."

Preview

Now you can build and view the page:

nikola auto -a 0.0.0.0 -p 8000

This will be hosted on CoCalc as described here as long as the nikola server is running. Only users authenticated to the CoCalc project can see it.

I have also put these commands into the Makefile, so one can do

make auto

Theming

To use my new themes, based on the zen-jinja theme, I needed to install that SASS plugin:

nikola plugin -i sass

and add the following to the conf.py file:

# conf.py ... NAVIGATION_LINKS = { DEFAULT_LANG: ( ('/index.html', 'Home', 'fa fa-home'), ('/archive.html', 'Archives', 'fa fa-folder-open'), ('/categories/index.html', 'Tags', 'fa fa-tags'), ('/rss.xml', 'RSS', 'fa fa-rss'), ('https://getnikola.com', 'About me', 'fa fa-user'), ('https://twitter.com/getnikola', 'My Twitter', 'fab fa-twitter'), ('https://github.com/getnikola', 'My Github', 'fab fa-github'), ) } ... # Compiler to process Sass files. SASS_COMPILER = 'sass' # A list of options to pass to the Sass compiler. # Final command is: SASS_COMPILER SASS_OPTIONS file.s(a|c)ss SASS_OPTIONS = []

Setup Blog (Old Version)

  • nikola init test_blog

    • Answer the questions

  • Follow the instructions on http://www.damian.oquanta.info/posts/ipython-plugin-for-nikola-updated.html for setting up your blog to work with IPython notebooks:

    • nikola install_theme zen-ipython

    • Add the following to your conf.py file:

      NAVIGATION_LINKS = { # Specific for the zen-ipython theme DEFAULT_LANG: ( ('/index.html', 'Home', 'icon-home'), ('/archive.html', 'Archives', 'icon-folder-open-alt'), ('/categories/index.html', 'Tags', 'icon-tags'), ('/rss.xml', 'RSS', 'icon-rss'), ('http://getnikola.com', 'About me', 'icon-user'), ('https://twitter.com/getnikola', 'My Twitter', 'icon-twitter'), ('https://github.com/getnikola', 'My Github', 'icon-github'), ) } ... POSTS = ( # Keep only one so that ``nikola new_post`` defaults to ipynb #("posts/*.rst", "posts", "post.tmpl"), #("posts/*.txt", "posts", "post.tmpl"), ("posts/*.ipynb", "posts", "post.tmpl"), ) PAGES = ( ("stories/*.rst", "stories", "story.tmpl"), ("stories/*.txt", "stories", "story.tmpl"), ("stories/*.ipynb", "stories", "story.tmpl"), ) ... THEME = "zen-ipython"

Generating Metadata

I have a bunch of notebooks I want to "blog" but need to generate metadata. Here is some code to do this programatically from the file information.

import os.path import time import nikola.utils POST_DIR = 'mmfblog/posts/' all_files = os.listdir(POST_DIR) new_posts = [_f for _f in all_files if _f.endswith('.ipynb') and _f[:-6] + ".meta" not in all_files] _template = '''\ .. title: {title} .. slug: {slug} .. date: {date} .. tags: .. link: .. description: .. type: text ''' TIME_FMT = '%Y-%m-%d %H:%M:%S %Z' doit = True for f in new_posts: subs = {} filename = os.path.join(POST_DIR, f) title, ext = os.path.splitext(f) slug = nikola.utils.slugify(unicode(title)) date = time.strftime(TIME_FMT, time.localtime(os.path.getmtime(filename))) subs.update(title=title, slug=slug, date=date) if slug != title: new_filename = os.path.join(POST_DIR, slug+ext) print("mv {0} {1}".format(filename, new_filename)) if doit: os.rename(filename, new_filename) meta_filename = os.path.join(POST_DIR, ".".join([slug, 'meta'])) meta = _template.format(**subs) print meta if doit: with open(meta_filename, 'w') as _mf: _mf.write(meta)
.. title: 3d-visualization .. slug: 3d-visualization .. date: 2014-10-24 23:51:28 UTC-07:00 .. tags: .. link: .. description: .. type: text .. title: coroutines .. slug: coroutines .. date: 2014-06-10 10:49:57 UTC-07:00 .. tags: .. link: .. description: .. type: text .. title: cylindrical-dvr .. slug: cylindrical-dvr .. date: 2013-10-13 21:49:16 UTC-07:00 .. tags: .. link: .. description: .. type: text .. title: dotproduct .. slug: dotproduct .. date: 2014-06-10 20:04:04 UTC-07:00 .. tags: .. link: .. description: .. type: text mv mmfblog/posts/Freeze (For cluster computing).ipynb mmfblog/posts/freeze-for-cluster-computing.ipynb .. title: Freeze (For cluster computing) .. slug: freeze-for-cluster-computing .. date: 2014-10-24 21:12:04 UTC-07:00 .. tags: .. link: .. description: .. type: text .. title: git-annex .. slug: git-annex .. date: 2014-06-03 02:25:07 UTC-07:00 .. tags: .. link: .. description: .. type: text .. title: interfaces .. slug: interfaces .. date: 2014-09-23 01:15:53 UTC-07:00 .. tags: .. link: .. description: .. type: text .. title: ipython-cluster .. slug: ipython-cluster .. date: 2014-10-16 21:52:55 UTC-08:00 .. tags: .. link: .. description: .. type: text mv mmfblog/posts/IPythonParallel.ipynb mmfblog/posts/ipythonparallel.ipynb .. title: IPythonParallel .. slug: ipythonparallel .. date: 2015-02-15 18:43:33 UTC-08:00 .. tags: .. link: .. description: .. type: text .. title: mayavi .. slug: mayavi .. date: 2014-03-24 02:44:58 UTC-07:00 .. tags: .. link: .. description: .. type: text mv mmfblog/posts/Mercurial Repository Conversion.ipynb mmfblog/posts/mercurial-repository-conversion.ipynb .. title: Mercurial Repository Conversion .. slug: mercurial-repository-conversion .. date: 2014-04-12 02:04:56 UTC-07:00 .. tags: .. link: .. description: .. type: text mv mmfblog/posts/MHD.ipynb mmfblog/posts/mhd.ipynb .. title: MHD .. slug: mhd .. date: 2014-05-05 15:42:04 UTC-07:00 .. tags: .. link: .. description: .. type: text .. title: optimization .. slug: optimization .. date: 2014-07-11 13:28:41 UTC-07:00 .. tags: .. link: .. description: .. type: text .. title: optimization_with_numba .. slug: optimization_with_numba .. date: 2013-11-26 12:45:25 UTC-08:00 .. tags: .. link: .. description: .. type: text .. title: profiling_python .. slug: profiling_python .. date: 2014-02-09 23:21:14 PST .. tags: .. link: .. description: .. type: text mv mmfblog/posts/PythonContainers.ipynb mmfblog/posts/pythoncontainers.ipynb .. title: PythonContainers .. slug: pythoncontainers .. date: 2015-02-15 04:15:06 PST .. tags: .. link: .. description: .. type: text mv mmfblog/posts/reStructuredText.ipynb mmfblog/posts/restructuredtext.ipynb .. title: reStructuredText .. slug: restructuredtext .. date: 2014-08-24 16:43:25 UTC-07:00 .. tags: .. link: .. description: .. type: text .. title: saddlepoint .. slug: saddlepoint .. date: 2014-04-24 17:11:55 UTC-07:00 .. tags: .. link: .. description: .. type: text .. title: software_carpentry .. slug: software_carpentry .. date: 2014-06-01 18:45:44 UTC-07:00 .. tags: .. link: .. description: .. type: text mv mmfblog/posts/Weizsacker Term.ipynb mmfblog/posts/weizsacker-term.ipynb .. title: Weizsacker Term .. slug: weizsacker-term .. date: 2014-09-29 00:27:25 UTC-07:00 .. tags: .. link: .. description: .. type: text mv mmfblog/posts/Widgets.ipynb mmfblog/posts/widgets.ipynb .. title: Widgets .. slug: widgets .. date: 2014-07-18 20:03:17 UTC-07:00 .. tags: .. link: .. description: .. type: text
f = "mmfblog/posts/optimization.ipynb" time.strftime(TIME_FMT, time.gmtime(os.path.getmtime(f)))
'2014-07-11 20:28:41 UTC-08:00'

Updating Nikola

If you need the latest development version of Nikola, you can install form sources:

conda install notebook nbformat nbconvert pillow cloudpickle docutils lxml mako natsort python-dateutil requests unidecode pip install git+https://github.com/getnikola/nikola.git webassets

I needed this before v7.3.1 to fix issue #1582.

Later I needed this before version v7.8.0 to deal with Ipython version 4.0 deprecations.

Later I needed this before this commit when sys.path was not including the root directory (where I keep a module that needs to be imported.)

import IPython.nbformat IPython.version_info IPython.nbformat.read from IPython.nbconvert.exporters import HTMLExporter
exportHtml = HTMLExporter() with open('test_blog/posts/test.ipynb') as f: nb_json = IPython.nbformat.reads(f.read(), IPython.nbformat.current_nbformat) (body, resources) = exportHtml.from_notebook_node(nb_json)

MathJax

If you want to use LaTeX in your notebook, be sure to add the following to your conf.py file.

# If you are using the compile-ipynb plugin, just add this one: MATHJAX_CONFIG = """ <script type="text/x-mathjax-config"> MathJax.Hub.Config({ tex2jax: { inlineMath: [ ['$','$'], ["\\\(","\\\)"] ], displayMath: [ ['$$','$$'], ["\\\[","\\\]"] ], processEscapes: true }, displayAlign: 'left', // Change this to 'center' to center equations. "HTML-CSS": { styles: {'.MathJax_Display': {"margin": 0}} } }); </script> """

Comments

I am trying the Disqus comment system since this was recommended in the Nikola manual. I had to create a Disqus account, and then register a new site to get the shortname. Some immediately apparent issues:

  • Disqus puts a lot of things in a global namespace. For example, the username (my usual handle was already taken) and when registering a new site, you need to guess a unique identifier <something>.disqus.com.

Website

Nikola can also be used for a static website. Here is the list of packages and versions I used when doing this:

!conda list -n blog
# packages in environment at /data/apps/conda/envs/blog: # # Name Version Build Channel aiohttp 3.6.2 py36h1de35cc_0 appnope 0.1.0 py36hf537a9a_0 argh 0.26.2 py36_0 argon2-cffi 20.1.0 py36haf1e3a3_1 async-timeout 3.0.1 py36_0 async_generator 1.10 py36h28b3542_0 attrs 20.2.0 py_0 babel 2.8.0 py_0 backcall 0.2.0 py_0 bleach 3.2.1 py_0 blinker 1.4 py36_0 brotlipy 0.7.0 py36haf1e3a3_1000 ca-certificates 2020.7.22 0 certifi 2020.6.20 py36_0 cffi 1.14.3 py36hed5b41f_0 chardet 3.0.4 py36_1003 cloudpickle 1.6.0 py_0 cryptography 3.1.1 py36hddc9c9b_0 decorator 4.4.2 py_0 defusedxml 0.6.0 py_0 docutils 0.16 py36_1 doit 0.32.0 py36_0 entrypoints 0.3 py36_0 freetype 2.10.2 ha233b18_0 ghp-import2 1.0.1 pypi_0 pypi icu 58.2 h0a44026_3 idna 2.10 py_0 idna_ssl 1.1.0 py36_0 importlib-metadata 1.7.0 py36_0 importlib_metadata 1.7.0 0 ipykernel 5.3.4 py36h5ca1d4c_0 ipython 7.16.1 py36h5ca1d4c_0 ipython_genutils 0.2.0 py36_0 jedi 0.17.2 py36_0 jinja2 2.11.2 py_0 jpeg 9b he5867d9_2 jsonschema 3.2.0 py36_1 jupyter_client 6.1.7 py_0 jupyter_core 4.6.3 py36_0 jupyterlab_pygments 0.1.2 py_0 lcms2 2.11 h92f6f08_0 libcxx 10.0.0 1 libedit 3.1.20191231 h1de35cc_1 libffi 3.3 hb1e8313_2 libiconv 1.16 h1de35cc_0 libpng 1.6.37 ha441bb4_0 libsodium 1.0.18 h1de35cc_0 libtiff 4.1.0 hcb84e12_1 libxml2 2.9.10 h3b9e6c8_1 libxslt 1.1.34 h83b36ba_0 lxml 4.5.2 py36h63b7cb6_0 lz4-c 1.9.2 hb1e8313_1 macfsevents 0.8.1 py36h1de35cc_0 mako 1.1.3 py_0 markdown 3.3 py36_0 markupsafe 1.1.1 py36h1de35cc_0 mistune 0.8.4 py36h1de35cc_0 mmf_setup 0.3.0 py_0 mforbes multidict 4.7.6 py36haf1e3a3_1 natsort 7.0.1 py_0 nbclient 0.5.0 py_0 nbconvert 6.0.7 py36_0 nbformat 5.0.7 py_0 nbstripout 0.3.9 pyh9f0ad1d_0 conda-forge ncurses 6.2 h0a44026_1 nest-asyncio 1.4.1 py_0 nikola 8.1.1 pypi_0 pypi notebook 6.1.4 py36_0 olefile 0.46 py36_0 openssl 1.1.1h haf1e3a3_0 packaging 20.4 py_0 pandoc 2.10.1 0 pandocfilters 1.4.2 py36_1 parso 0.7.0 py_0 pathtools 0.1.2 py_1 pexpect 4.8.0 py36_0 pickleshare 0.7.5 py36_0 piexif 1.1.3 py_2 conda-forge pillow 7.2.0 py36ha54b6ba_0 pip 20.2.3 py36_0 pipdeptree 1.0.0 pypi_0 pypi prometheus_client 0.8.0 py_0 prompt-toolkit 3.0.7 py_0 ptyprocess 0.6.0 py36_0 pycparser 2.20 py_2 pygments 2.7.1 py_0 pyopenssl 19.1.0 py_1 pyparsing 2.4.7 py_0 pyrsistent 0.17.3 py36haf1e3a3_0 pyrss2gen 1.1 pypi_0 pypi pysocks 1.7.1 py36_0 python 3.6.12 h26836e1_2 python-dateutil 2.8.1 py_0 python-hglib 2.6.1 py_0 mforbes pytz 2020.1 py_0 pyyaml 5.3.1 py36haf1e3a3_1 pyzmq 19.0.2 py36hb1e8313_1 readline 8.0 h1de35cc_0 requests 2.24.0 py_0 send2trash 1.5.0 py36_0 setuptools 50.3.0 py36h0dc7051_1 six 1.15.0 py_0 smartypants 2.0.0 pyh9f0ad1d_2 conda-forge sqlite 3.33.0 hffcf06c_0 terminado 0.8.3 py36_0 testpath 0.4.4 py_0 tk 8.6.10 hb0a8c7a_0 tornado 6.0.4 py36h1de35cc_1 traitlets 4.3.3 py36_0 typing_extensions 3.7.4.3 py_0 typogrify 2.0.7 py36_0 conda-forge unidecode 1.1.1 py_0 urllib3 1.25.10 py_0 watchdog 0.10.3 py36haf1e3a3_0 wcwidth 0.2.5 py_0 webencodings 0.5.1 py36_1 wheel 0.35.1 py_0 xz 5.2.5 h1de35cc_0 yaml 0.2.5 haf1e3a3_0 yapsy 1.12.2 py_0 conda-forge yarl 1.6.0 py36haf1e3a3_0 zeromq 4.3.2 hb1e8313_3 zipp 3.3.0 py_0 zlib 1.2.11 h1de35cc_3 zstd 1.4.5 h41d2c2f_0
$ nikola init forbes-group Creating Nikola Site ==================== This is Nikola v7.8.3. We will now ask you a few easy questions about your new site. If you do not want to answer and want to go with the defaults instead, simply restart with the `-q` parameter. --- Questions about the site --- Site title [My Nikola Site]: Forbes Group Website Site author [Nikola Tesla]: Michael McNeil Forbes Site author's e-mail [[email protected]]: [email protected] Site description [This is a demo site for Nikola.]: Forbes Group Research Site Site URL [https://example.com/]: http://swan.physics.wsu.edu/forbes/ Enable pretty URLs (/page/ instead of /page.html) that don't need web server configuration? [Y/n] --- Questions about languages and locales --- We will now ask you to provide the list of languages you want to use. Please list all the desired languages, comma-separated, using ISO 639-1 codes. The first language will be used as the default. Type '?' (a question mark, sans quotes) to list available languages. Language(s) to use [en]: Please choose the correct time zone for your blog. Nikola uses the tz database. You can find your time zone here: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones Time zone [America/Vancouver]: Current time in America/Vancouver: 00:53:13 Use this time zone? [Y/n] --- Questions about comments --- You can configure comments now. Type '?' (a question mark, sans quotes) to list available comment systems. If you do not want any comments, just leave the field blank. Comment system: disqus You need to provide the site identifier for your comment system. Consult the Nikola manual for details on what the value should be. (you can leave it empty and come back later) Comment system site identifier: forbes-group That's it, Nikola is now configured. Make sure to edit conf.py to your liking. If you are looking for themes and addons, check out https://themes.getnikola.com/ and https://plugins.getnikola.com/. Have fun! [2017-01-21T08:53:36Z] INFO: init: Created empty site at forbes-group.

Now version control the generated files:

cd forbes-group hg init hg add hg com -m "Initial Commit"

We now switch the theme to zen-ipython which requires installing some themes:

$ nikola install_theme zen-ipython [2017-01-21T09:01:18Z] WARNING: Nikola: Cannot load theme "zen-ipython", using 'bootstrap3' instead. [2017-01-21T09:01:18Z] INFO: theme: Downloading 'https://themes.getnikola.com/v7/zen-ipython.zip' [2017-01-21T09:01:19Z] INFO: theme: Extracting 'zen-ipython' into themes/ [2017-01-21T09:01:19Z] NOTICE: theme: This theme has a sample config file. Integrate it with yours in order to make this theme work! Contents of the conf.py.sample file: NAVIGATION_LINKS = { DEFAULT_LANG: ( ('/index.html', 'Home', 'icon-home'), ('/archive.html', 'Archives', 'icon-folder-open-alt'), ('/categories/index.html', 'Tags', 'icon-tags'), ('/rss.xml', 'RSS', 'icon-rss'), ('https://getnikola.com', 'About me', 'icon-user'), ('https://twitter.com/getnikola', 'My Twitter', 'icon-twitter'), ('https://github.com/getnikola', 'My Github', 'icon-github'), ) } [2017-01-21T09:01:19Z] INFO: theme: Downloading 'https://themes.getnikola.com/v7/zen-jinja.zip' [2017-01-21T09:01:19Z] INFO: theme: Extracting 'zen-jinja' into themes/ [2017-01-21T09:01:19Z] NOTICE: theme: This theme has a sample config file. Integrate it with yours in order to make this theme work! Contents of the conf.py.sample file: NAVIGATION_LINKS = { DEFAULT_LANG: ( ('/index.html', 'Home', 'icon-home'), ('/archive.html', 'Archives', 'icon-folder-open-alt'), ('/categories/index.html', 'Tags', 'icon-tags'), ('/rss.xml', 'RSS', 'icon-rss'), ('https://getnikola.com', 'About me', 'icon-user'), ('https://twitter.com/getnikola', 'My Twitter', 'icon-twitter'), ('https://github.com/getnikola', 'My Github', 'icon-github'), ) } [2017-01-21T09:01:19Z] NOTICE: theme: Remember to set THEME="zen-ipython" in conf.py to use this theme.

Organization

In addition to static content, I would like to have posts on my webpage describe research, projects, etc. These can be organized in "sections" by placing them in appropriate directories in posts and then specifying the output directories in the POSTS variable in conf.py. One can then filter them, for example, with {% for post in posts if post.section_name().lower() == "highlights" %} in a template.

I use this with a special template templates/research_highlights_list.tmpl to format the research highlights on my index page.

Customization

Themes

Here we collect some notes about how themes work and can be customized. Themes are stored in the themes directory. For example, themes/zen-ipython. I have a separate repo for custom themes which I include in _ext/forbes_group_website_theme and then link into the themes.

A Nikola site can be customized in several ways:

  1. conf.py: Many customizations appear in this file, including some that might not be obvious:

    • INDEX_READ_MORE_LINK: This variable is used to customize the "Read more" link. We used this to modify these to look like a button matching the "Editorial" style.

    • NAVIGATION_LINKS: These appear in the side-bar, so you can specify the icons here (from Font Awesome if you use the zen-* themes).

    • POSTS/PAGES: Here you can specify which template files are used for posts and pages.

  2. Modify the templates. To do this, I found it easiest to copy all of the inherited template files into a new theme's template directory so I can see what is happening. Here are some important templates:

    • index.tmpl: Used for the index page collecting your blog posts.

    • post.tmpl: Used for the individual blog posts. This can be selected in POSTS variable of conf.py. These can also be controlled in the actual posts with the template variable.

    • story.tmpl: Used for static pages. This can be selected in PAGES variable of conf.py. These can also be controlled in the actual posts with the template variable.

zen-ipython

This directory contains a file themes/zen-ipython/parent that specify the parent theme - in this case themes/zen-jinja. Thus, to find assets, etc. one might need to look in several places. Here is the full hierarchy for the themes/zen-ipython theme:

The last two are pre-installed themes whose location on disk can be found using the specified command. I copied them into my site folder for easy reference.

Icons

The zen-ipython theme uses icons from Font Awesome. To see the available icons and names, look at the icon list for the appropriate version (version 3.2.1 here). You can then specify the appropriate fonts in conf.py.

Less/Sass

The zen themes use [less], which they tell you in the Readme.md file can be enabled by setting USE_BUNDLES = False in conf.py after installing the lessc compiler and nikola plugin -i less, however, you also need to rename/remove themes/zen-jinja/assets/css/main.cssmain.css file or else you will get the following error (since it is also built with lessc...)

ERROR: Two different tasks can't have a common target.'output/assets/css/main.css' is a target for copy_assets:output/assets/css/main.css and build_less:output/assets/css/main.css.

Targets for less are specified in the file themes/zen-jinja/less/targets.

I am using a custom theme which provides sass stylesheets. This requires installing the sass compiler and then one can use the sass plugin to generate appropriate CSS files. This theme was inheriting from the zen-jinja theme and so the main.css file was clashing with the previous file specified in themes/zen-jinja/less/targets. To resolve this, I added a custom less directory in the new theme with an empty themes/html5up-editorial/less/targets file.

Jinja2

Here are some tips for working with Jinja2 (see the references section below for details.)

  • You can see the value of variables using code like this: {{ post.tags|pprint }} or if you need to {{ post.__dict__.keys()|pprint }}.

Images

One issue I had is how to include images in posts. The usual approach for notebooks is to include the images in a file relative to the notebook. For example, if the files are in a folder images then one can refer to this as

![Image](images/eli-francis-100644.jpg)

There are two issues with this approach:

  1. Nikola will not copy these images to the output. This can be overcome by storing the files in the images folder at the root of your project, and then specifying where these should appear in the output by specifying IMAGE_FOLDERS = {'images': 'images'} (the default value) in your conf.py file. You can then symlink this file to your posts directory.

  2. The second issue is that this image will produce a link like this in the output:

    /highlights/prerequesites/images/eli-francis-100644.jpg

    I.e. this is a link relative to the post which would require a complicated set of non-standard IMAGE_FOLDERS = {'images': 'images'} mappings.

The standard Nikola solution is to place the images in a images/ folder in the top level of your project as above with IMAGE_FOLDERS = {'images': 'images'} in your conf.py file. Then refer with an absolute address:

![Image](/images/eli-francis-100644.jpg)

Unfortunately, this breaks in the notebook rendering which cannot see higher-level directories. For example, if you start the Jupyter notebook server in the same directory as the post, then one would have to use the following absolute path in the notebook in order to see the image:

![Image](/tree/images/eli-francis-100644.jpg)

This location could then be hacked with Nikola using IMAGE_FOLDERS = {'images': 'tree/images'}.

Solutions

I see three possible solutions:

  1. Use the modified "tree" hack above:

    1. Place images in

      images/eli-francis-100644.jpg
    2. Symlink this to an images folder next to your posts.

    3. Refer to your images as

      ![Image](/tree/images/eli-francis-100644.jpg)
    4. Use the following mapping in your conf.py file:

      IMAGE_FOLDERS = {'images': 'tree/images'}
  2. Mirror the structure of your posts in your images directory. For example, if your posts are in posts/highlights/my_post.ipynb, then:

    1. Place images in

      images/highlights/my_post/images/eli-francis-100644.jpg
    2. Symlink these images to your posts directory:

      link -s ../../images/highlights/my_post/images .
    3. Refer to images with a relative path:

      ![Image](images/eli-francis-100644.jpg)
    4. Map the images to the root level so they are included in the relative path:

      IMAGE_FOLDERS = {'images': ''}
  3. Use the Nikola recommend approach of linking to an absolute path and extend the jupyter notebook server to properly redirect these requests.

    1. Place images in

      images/eli-francis-100644.jpg
    2. Refer to your images using an absolute path

      ![Image](/images/eli-francis-100644.jpg)
    3. Run jupyter notebook from the top level (where images is) after installing and loading my notebook server extension mmf_nikola_nbserver_extension (this is done in the file jupyter_notebook_config.py:

      pip install -e mmfsite/mmf_nikola_nbserver_extension jupyter serverextension enable mmf_nikola_nbserver_extension

I am presently using the last approach.

Notebook Conversion

The notebooks are converted to HTML through the nbcovert process. This can be customized through templates or providing plugins.

  • The 'IPYNB_CONFIG' variable in conf.py is passed to the HTMLExporter. To see possible options, run jupyter nbconvert --help-all. For example, I have added a preprocessor mmf_nikola_nbserver_extension.ArXivLinks which replaces references to preprints with a URL. This is enabled as follows in conf.py:

    IPYNB_CONFIG = { 'Exporter': { 'preprocessors': ['mmf_nikola_nbserver_extension.ArXivLinks'] } }

Plugins

Sometimes you just can't seem to do what you want with the existing tools. One way of extending Nikola is to use a plugin. First check the Nikola Plugins to see if you can find something, otherwise you can create your own in the plugins directory. To add a simple reStructuredText directive, for example, you can install and copy the book_figure plugin, which is a fairly simple plugin that creates a book_figure directive:

nikola plugin -i book_figure mv plugins/book_figure plugins/mmf

then edit the plugin.

There are also some useful plugins: here is a list of the plugins I sometimes use:

nikola plugin -i less nikola plugin -i sass nikola plugin -i localsearch

References

People who have customized Nikola and describe the process:

Official Documentation:

Jinja Templating:

Some nice HTML and CSS templates:

  • HTML5 UP: Some nice, responsive, and clean HTML and CSS templates. We have tried to massage one of these into a website.