Welcome to KBasix

Introduction

This is the KBasix CMS system. It provides basic authentication and file-management capabilities, together with a simple set of ACL controls. It also supports mathematical notation via MathJax:

$$e^{i \pi} + 1 = 0$$

Basic features

  • Internal authentication (LDAP too)
  • File uploads, with quotas
  • Flexible, human-readable metadata
  • Data manipulation (e.g. video transcoding)
  • Supports user and group ACLs (read-only)
  • Does not use cookies
  • Javascript is optional (but needed for MathJax)
  • File-based database
  • Command-line backend
  • Written in Python
  • BSD license

Requirements

  • RHEL 6 or CentOS 6 (tested under Red Hat Enterprise Linux)
  • An Apache webserver
  • The following modules: mod_python (from EPEL) and mod_ssl
  • The Python programming language
  • The zsh shell is recommended

Download

Two packages are offered, with and without MathJax:

Installation

Note: KBasix is BETA software, it's currently being tested and developed. Use at your own risk! Bug and security reports welcomed at marco@math.utoronto.ca.

Extracting the downloaded bundle by means of tar -jxvf kbasix_m.tbz2 yields the following file hierarchy:

        kbasix
          |-data
          |-files
          |   |_shared
          |       |_local
          |-logs
          |-sys
          |_web
             |-cms
             |-MathJax
             |_users

(don't forget to do chown -R webuser:webgroup kbasix, where webuser and webgroup are the user and group Apache runs as). This example shows the contents of the MathJax bundle, but it's optional and can be disabled from defs.py.

The data directory is a staging area for processing files (for example, transcoding video, which requires new files to be temporarily stored therein as they are being created). All user-uploaded files are kept in files, and logs is self-explanatory. The sys directory contains the user and group information files in JSON format. None of these should be directly accessible through the web server. The web directory, on the other hand, should be made DocumentRoot in httpd.conf (e.g. DocumentRoot /www/kbasix/web). Furthermore, the following two directives should be set: Options SymLinksifOwnerMatch and AllowOverride All. KBasix proper is stored in cms. Files which the users have shared with the world are symlinked within subdirectories inside users so that, for example, user "Sora" can make her files accessible at https://www.mykbsite.org/users/Sora (assuming the site is hosted at www.mykbsite.org).

You can now configure KBasix by editing the defs.py file inside cms (make sure not to leave any backup files behind e.g. defs.py~). The file is documented, but in the simplest case suffice to set the variables in the QUICK CONFIGURATION segment near the top of the file. In particular, make sure change kbasix_root_, kbasix_url_, and kbasix['email_from_'] to their proper values (in this example they'd be /www/kbasix, https://www.mykbsite.org, and (say) kbasix.admin@mykbsite.org respectively).

Finally, edit the CSS/HTML files to taste (probably not needed for now). For extra security it's a good idea to narrow permissions and make some files and directories immutable. Running the following (as root) is recommended:

       zsh -f
       chmod 600 kbasix/**/*(.)
       chmod 600 kbasix/**/.htaccess
       chmod 700 kbasix/**/*(/)
       chmod 730 kbasix/data
       chmod 710 kbasix/files
       chmod 710 kbasix
       chattr +i kbasix/**/.htaccess
       chattr +i kbasix/web/*(.)
       chattr +i kbasix/logs
       chattr +i kbasix/web/MathJax
       chattr +i kbasix/web
       chmod 700 kbasix/web/cms/aux_video.py
       chattr -R +i kbasix/web/cms
       chattr +i kbasix
       chmod 700 kbasix/sys/kbchk.py

You should verify the file attributes and permissions by running the kbchk.py command located inside the sys directory.

Data manipulation (for example, to transcode files) is done by a non-privileged user who only has access to the files which are being processed. This guarantees that a rogue module will not compromise the entire KBasix system. For example, the aux_video.py uses ffmpeg as its transcoding backend, but it is run by the videodm user. If for any reason there is an exploit in ffmpeg it can only run with videodm privileges, which means that the KBasix system and user files should remain intact (as they are owned by webuser). Note that other files being concurrently processed by videodm will be vulnerable, but the risk is small, and can even be nullified by turning on process queuing (at the expense of performance). One important caveat is of course if there is a privilege escalation bug which might allow videodm to access webuser files bypassing directory permissions, but this would be a serious OS vulnerability beyond the scope of KBasix to deal with.

Currently the only data manipulator module available is aux_video.py, which can transcode video files into web-friendly formats using ffmpeg. In order to enable this functionality the user under which the module runs must be set up as follows (as root):

       zsh -f
       useradd -c "KBasix video data manager" -M -g webgroup -s /sbin/nologin videodm
       cd /etc/sudoers.d
       echo 'Defaults:webuser !requiretty' > kbasix
       echo 'webuser  ALL=(videodm) NOPASSWD:/usr/local/bin/ffmpeg, /bin/chmod' >> kbasix
       chmod 400 kbasix

Installation instructions for ffmpeg can be found at http://ffmpeg.org/trac/ffmpeg/wiki/CentosCompilationGuide, but a non-documented summary is shown below (be aware of using wildcards when running yum, in the case below only the main channel and "optional" repository are active). In order to keep a clear separation between Red Hat and user-compiled code the latter is stored under /usr/local/software (compilations can, furthermore, be all done as a non-privileged user, as long as all permissions are set accordingly):

       zsh -f
       umask 022
       yum remove ffmpeg x264 x264-devel libvpx'*'
       yum install gcc git make nasm pkgconfig wget libogg'*' libtheora'*' libvorbis'*' zlib'*'
       mkdir /usr/local/src/ffmpeg-sources
       mkdir -p /usr/local/software/{vo-aacenc-0.1.2,yasm-1.2.0,x264,lame-3.99.5,libvpx,ffmpeg}
       cd /usr/local/src/ffmpeg-sources
       wget http://www.tortall.net/projects/yasm/releases/yasm-1.2.0.tar.gz
       tar xzvf yasm-1.2.0.tar.gz
       cd yasm-1.2.0
       ./configure --prefix=/usr/local/software/yasm-1.2.0
       make
       make install
       for i in yasm vsyasm ytasm
        ln -s /usr/local/software/yasm-1.2.0/bin/${i} /usr/local/bin/${i}
       cd -
       wget 'http://sourceforge.net/projects/opencore-amr/files/vo-aacenc/vo-aacenc-0.1.2.tar.gz/download'
       tar xzvf vo-aacenc-0.1.2.tar.gz
       cd vo-aacenc-0.1.2
       ./configure --prefix=/usr/local/software/vo-aacenc-0.1.2
       make
       make install
       echo '/usr/local/software/vo-aacenc-0.1.2/lib' > /etc/ld.so.conf.d/vo-aacenc-0.1.2.conf
       ldconfig
       cd -
       git clone git://git.videolan.org/x264
       cd x264
       ./configure --enable-static --prefix=/usr/local/software/x264
       make
       make install
       cd -
       wget http://downloads.sourceforge.net/project/lame/lame/3.99/lame-3.99.5.tar.gz
       tar xzvf lame-3.99.5.tar.gz
       cd lame-3.99.5
       ./configure --disable-shared --enable-nasm --prefix=/usr/local/software/lame-3.99.5
       make
       make install
       cd -
       git clone http://git.chromium.org/webm/libvpx.git
       cd libvpx
       ./configure --prefix=/usr/local/software/libvpx
       make
       make install
       cd -
       git clone git://source.ffmpeg.org/ffmpeg
       cd ffmpeg
       ./configure --enable-gpl --enable-libmp3lame --enable-libtheora --enable-libvo-aacenc \
                   --enable-libvorbis --enable-libvpx --enable-libx264 --enable-version3 \
                   --extra-libs=-L/usr/local/software/vo-aacenc-0.1.2/lib \
                   --extra-libs=-L/usr/local/software/x264/lib \
                   --extra-libs=-L/usr/local/software/libvpx/lib \
                   --extra-libs=-L/usr/local/software/lame-3.99.5/lib \
                   --extra-cflags=-I/usr/local/software/vo-aacenc-0.1.2/include \
                   --extra-cflags=-I/usr/local/software/x264/include \
                   --extra-cflags=-I/usr/local/software/libvpx/include \
                   --extra-cflags=-I/usr/local/software/lame-3.99.5/include \
                   --prefix=/usr/local/software/ffmpeg
       make
       make install
       for i in ffmpeg ffprobe ffserver
        ln -s /usr/local/software/ffmpeg/bin/${i} /usr/local/bin/${i}

To access the CMS make sure the Apache web server is up and running with mod_ssl and mod_python enabled.

       rpm -Uvh http://www.muug.mb.ca/pub/epel/6/i386/epel-release-6-8.noarch.rpm
       yum install mod_ssl mod_python
KBasix will only work through an encrypted connection, so having an SSL certificate is nice (but not required). What is important is that the ssl.conf file be properly configured. Assuming that's been done, and that the URL is https://www.mykbsite.org, it is now necessary to configure python.conf as follows:

        <Directory /www/kbasix/web/cms>
          AddHandler python-program .py
          PythonHandler mod_python.publisher
          PythonDebug Off
        </Directory>

(to aid debugging on a non-production server one can set PythonDebug On). Restart Apache and if all goes well KBasix should be reachable at: https://www.mykbsite.org

Run tail -F on the KBasix log in the logs directory to keep track of what's going on. The default level in defs.py is debug. Click on the Register button at the top of the CMS to create a KBasix account.

Migration

If migrating to a newer version of KBasix perform the following procedure. Note that ongoing data manipulation processes will have to be restarted by the users (they will be notified) — the alternative is to stop the web server and await their completion. Unfortunately there is no way yet to gracefully stop ongoing data manipulation processes.

  1. Install KBasix as explained above into a new directory (say, /www/kbasix.new). Edit defs.py (the old defs.py might not work while KBasix remains in BETA).
  2. Stop the web server, and kill any data manipulator processes: /etc/init.d/httpd stop and maybe killall -s9 -u videodm
  3. Copy the accounts information: cp -pf /www/kbasix/sys/*.json /www/kbasix.new/sys/
  4. Copy the user files: rsync -avc /www/kbasix/files/ /www/kbasix.new/files/
  5. Copy the world-shared links: rsync -avc /www/kbasix/web/users/ /www/kbasix.new/web/users/
  6. Copy the transactions log (if any): cp -p /www/kbasix/data/transactions.log /www/kbasix.new/data/
  7. Run: /www/kbasix.new/sys/kbchk.py. This will check permissions, update the necessary metadata, and notify users of cancelled transactions.
  8. Retire the old installation: chattr -i /www/kbasix ; mv /www/kbasix /www/kbasix.old
  9. Enable the new installation: chattr -i /www/kbasix.new ; mv /www/kbasix.new /www/kbasix ; chattr +i /www/kbasix
  10. Start the web server: /etc/init.d/httpd start

If need be, one can back-out of the update as so:

  1. /etc/init.d/httpd stop
  2. killall -s9 -u videodm
  3. mv /www/kbasix /www/kbasix.hide
  4. mv /www/kbasix.old /www/kbasix
  5. /www/kbasix/sys/kbchk.py
  6. /etc/init.d/httpd start

Note that unless changes made after the update are merged back, they will be lost.

Basic operation

Starting KBasix is simply a matter of starting the web server as explained above. Stopping KBasix is done by stopping the web server and killing the processes being run by the data manipulators. kbchk.py should always be run after stopping KBasix, whether it was deliberate or after an unexpected shutdown.

KBasix Manual

There isn't a user or administrator guide (yet). Overall configuration is done by editing defs.py (which is fairly well documented). The entire KBasix CMS can easily be relocated by simply moving the kbasix/ hierarchy (and changing the kbasix_root_ and kbasix_url_ settings in defs.py). A KBasix installation can be backed up with a simple tar or rsync of kbasix/.

KBasix has no concept of an administrator at the GUI level (although one can certainly be written). Low-level administration is through the command line. To use it, cd into kbasix/web/cms and run python. The two main administration modules are manage_users and manage_kbasix. A brief sample session is as follows:

     # whoami
     webuser
     # pwd
     /www/kbasix/web/cms
     # python
     Python 2.6.6 (r266:84292, May  1 2012, 13:52:17)
     [GCC 4.4.6 20110731 (Red Hat 4.4.6-3)] on linux2
     Type "help", "copyright", "credits" or "license" for more information.
     >>> import manage_users
     >>> import manage_kbasix
     >>> manage_users._finger()
     [u'acky', u'nana', u'sora', u'yuma']
     >>> manage_kbasix._finger('yuma')
     {u'last_login': 1345738231.8814499,
      u'login_name': u'yuma',
      u'quota': 1048576,
      u'registered': 1344365184.5450959,
      u'user_email': u'yuma-a@hello.av.jp',
      u'user_name': u'YumA'}
     >>> print(manage_users._mod.__doc__)
     Modify a user or a group.
     
            (OK, status) = _mod(name, settings, is_type='account')
      
         'name' is either a login or group name. The 'is_type' parameter can
         be 'account' or 'group'. User/group properties are modified via the
         'settings' dictionary, the keys of which correspond to those from
         the '_user_add'/'_group_add' functions (unknown keys are ignored).
         Return is (bool, str).
     
     >>>

Notable functions available from manage_users are:

       _user_add
       _authenticate
       _group_add
       _del
       _mod
       _info
       _finger

And from manage_kbasix:

      _account_add
      _account_mod
      _account_del
      _account_info
      _finger

Information about these functions can be easily obtained by via .__doc__. KBasix is offered with icons by Mark James.

Changelog

Latest Release

New

Changes

Released