Puppet RSpec set up and testing with Puppet Forge module dependencies

Puppet & RSpec

Reference: https://github.com/rodjek/rspec-puppet

Link to example code

As at the 21^st April 2017, Puppet and RSpec testing is still an issue, with .fixtures.yml not full functioning as it should, that is loading dependencies into the spec/fixtures directory.

Even when you place your .fixtures.yml in the root of your project directory, which is to say the module where the spec directory lives along with the modules and manifest, and you run the puppet-spec-init or rspec or rake commands your fixtures file is ignored, and the only module linked is the current module.

It is essential that after creating your module with the puppet module generate command you will get the necessary layout for your puppet module.  Once you start to build your module you’ll eventually want to add some code spec tests to ensure that someone doesn’t break the existing code.

ServerSpec tests can be run independently of your code, so the key difference is that RSpec is there to perform tests on your code and not the final server build.

To fix this means you still need to deal with all your module dependencies manually, so here are the steps.

Example project space;

The Rakefile

require 'rspec-puppet/rake_task'

begin

  if Gem::Specification::find_by_name('puppet-lint')

    require 'puppet-lint/tasks/puppet-lint'

    PuppetLint.configuration.send('disable_autoloader_layout')

    PuppetLint.configuration.ignore_paths = ["spec/**/*.pp", "vendor/**/*.pp"]

    task :default => [:rspec, :lint]

  end

rescue Gem::LoadError

  task :default => :rspec

end

This file controls what will happen when the tests run, such as in this example a linter will run against your code first, and certain files will be ignores by the linter as they are not puppet code.

The spec_helper.rb file

This file defines other files required to help perform the testing, and where the files to test can be located.

require 'rspec-puppet/spec_helper' 

fixture_path = File.expand_path(File.join(__FILE__, '..', 'fixtures')) 

RSpec.configure do |c| 
  c.module_path = File.join(fixture_path, 'modules') 
  c.manifest_dir = File.join(fixture_path, 'manifests') 
  c.environmentpath = File.join(Dir.pwd, 'spec') 
end

Preparing the module dependencies

First let’s take a look at a sample directory layout of what is required;

mysqlmodule

├── Gemfile

├── Gemfile.lock

├── manifests

│   └── init.pp

├── Rakefile

├── spec

│   ├── classes

│   │   └── mysqltest_spec.rb

│   ├── defines

│   ├── fixtures

│   │   ├── manifests

│   │   │   └── site.pp

│   │   └── modules

│   │       ├── mysql

│   │       │   ├── lib -> ../../../../../mysql/lib

│   │       │   ├── manifests -> ../../../../../mysql/manifests

│   │       │   └── templates -> ../../../../../mysql/templates

│   │       ├── mysqlmodule

│   │       │   ├── manifests -> ../../../../manifests

│   │       │   └── templates -> ../../../../templates

│   │       ├── staging

│   │       │   ├── files -> ../../../../../staging/files

│   │       │   ├── libs -> ../../../../../staging/libs

│   │       │   └── manifests -> ../../../../../staging/manifests

│   │       └── stdlib

│   │           ├── lib -> ../../../../../stdlib/lib

│   │           ├── manifests -> ../../../../../stdlib/manifests

│   │           └── types -> ../../../../../stdlib/types

│   ├── functions

│   ├── hosts

│   └── spec_helper.rb

└── templates

You’ll notice that in this module the user has made use of PuppetForge modules for MySQL, which in itself has other dependencies on Staging and StdLib. For RSpec to work these fixtures need to be available during the test to enable checking of your own modules code.  For example;

it { is_expected.to contain_class_mysql__db(‘mydb’) }

This would look for  mysql::db{‘mydb’:} in your code.

Since the .fixture.yml is not being read at the moment, you should manually add the dependencies to your 

spec/fixtures/modules directory.  Let’s take the MySQL module.

In our modules directory, that is where we would see mysqlmodule if listed, we would ensure that we have done a puppet module install of those modules require, and making sure we note the dependencies that have been downloaded too.

Here is the layout of the MySQL puppetlabs/mysql module;

mysql

├── CHANGELOG.md

├── checksums.json

├── CONTRIBUTING.md

├── examples/

├── Gemfile

├── lib/

├── LICENSE

├── manifests/

├── metadata.json

├── NOTICE

├── Rakefile

├── README.md

├── spec/

├── templates/

└── TODO

In our fixtures directory under modules we will create a directory called mysql and within that directory you will create symlinks back to the essential directories and files that make up the module.  In this case we will symlink to;

- lib

- manifests

- templates

We would then do the same for the other 2 dependent modules stdlib and staging to ensure that our tests will work with all elements.

Overcoming variable issues

When running rspec after configuring your module dependencies, you may then find that your test still fails.  Right near the top of the output you might see something like;

F

Failures:

  1) mysqlmodule should contain File_line[/etc/my.cnf] with bind-address => "0.0.0.0"

     Failure/Error: it {is_expected.to contain_file_line('/etc/my.cnf').with('bind-address' => '0.0.0.0')}

     

     Puppet::PreformattedError:

       Evaluation Error: Unknown variable: '::osfamily'. at /home/steve/web/puppet/modules/mysqlmodule/spec/fixtures/modules/mysql/manifests/params.pp:36:8 on node tpslaptop

From this output we are interested in the line that comes after Puppet::PreformatedError which states that the variable ::osfamily is missing.  This is actually a Puppet Facter variable, and would normally be collected by the Puppet Master during the compilation of the Puppet catalog.  We therefore need to define all of the values for the Facts in our classes spec test.

Our spec test file is called mysqltest_spec.rb in spec/classes, and originally looked like;

require 'spec_helper'

describe 'mysqlmodule' do

  it {is_expected.to contain_file_line('/etc/my.cnf').with('bind-address' => '0.0.0.0')}

end

  

The test is using the file_line type from the mysql module, and we need to tell our test to set the osfamily facter variable before this is run.  The change to the code that will allow the variable to be set is;

let(:facts) do

  { :osfamily => RedHat}

end

This piece of code is added within the describe block before the tests, and we have provided a value.

We would need to add each variable into the block as they appear during our tests.  For example, the next one to pop up would be ::operatingsystem, so we would have the following code;

require 'spec_helper'

describe 'mysqlmodule' do

  let(:facts) do

{

  :osfamily => 'RedHat' ,

      :operatingsystem => 'RedHat' 

    }

  end

  it {is_expected.to contain_file_line('/etc/my.cnf').with('bind-address' => '0.0.0.0')}

end

After running through the test we eventually find that to test the file_line type for the MySQL module we would need the following;

describe 'mysqlmodule' do

  let(:facts) do

    { :osfamily => 'RedHat',

    :operatingsystemmajrelease => '7.2',

    :root_home => '',

    :puppetversion => '4.10.0',

    :operatingsystem => 'RedHat' }

  end

  it {is_expected.to contain_file_line('Edit my.cnf')}

end

This will finally result in your module being checked, or at least the file_line type;

.

Finished in 2.24 seconds (files took 0.93215 seconds to load)

1 example, 0 failures

Now you repeat for the other modules.

Getting installed services in RHEL

Today I was asked how you could discover those services that are installed on a system.

For packages installed today;
rpm -ql $(rpm -qa --last | 
grep "$(date +'%a %d %b %Y')" | 
awk '{print $1}') | 
grep systemd | 
grep service | 
awk -F"/" '{print $NF}' | 
grep -v '@'

From the beginning of time;
rpm -ql $(rpm -qa --last | 

awk '{print $1}') | 
grep systemd | 
grep service | 
awk -F"/" '{print $NF}' | 
grep -v '@'

From using these you can then ask which are enabled on start up;

systemctl list-unit-files --type=service |

grep enabled |

egrep $(

  rpm -ql $(rpm -qa --last | 

  grep "$(date +'%a %d %b %Y')" | 

  awk '{print $1}') | 

  grep systemd | 

  grep service | 

  awk -F"/" '{print $NF}' | 

  grep -v '@' | 

  tr '\n' '|' | 

  sed 's/|$//'

)

Dead Icons on Mac OS X

Having used a Mac for the last 6 months again (how annoying), I went through the process of clearing it up ready for the next person to use (as I'd never personally own an Apple Mac, since I was the person who coined the term "Crapple Mac").

However, in the process of working out how to remove applications that were not installed through iTunes (I have no intention of putting my credit card details into a laptop I don't own, or to Apple), most of my applications were installed via brew.

I had the following shell script to install, search or delete my applications;

#!/bin/bash

if (( $# < 2 ))

then

echo "SYNTAX: $0 action package" >&2

echo "Where action is one of (search|install|uninstall)" >&2

exit 1

fi

action=$1

shift

package="$@"

case $action in

'install')

brew cask $action $package --force

;;

        remove|erase)

                brew cask uninstall $package

;;

*)

brew cask $action $package

;;

esac

So when coming to delete the applications were not always easy to remove with brew, as some were downloaded over the Internet.

So if brew odes not remove the application you can try one of the following 3 options;

1. The simple method of clicking on Launchpad and then click and hold on the application to remove.  All the icons will then wobble.  If you can remove the application here then an X will appear against the application.  Clicking it will allow you to remove it.
2. Open the application.  Then bring up the menu for the application in the dock so that you can "show in finder".  Then quit the application (don't just close, you need to quit it) and then drag the .app from Finder into the waste basket and then empty trash.
3. This final one will help remove application Icons where the application has gone, but the icon refuses to budge from Launchpad.  The following describes how to do this.

To remove an icon or application still showing in Launchpad that could not be remove using steps 1 or 2 do the following;

1. To find the application and see if it is in Launchpad run the following command

for x in $(find /private/var/folders -name *launchpad*)

do

  sqlite3 $x/db/db "SELECT * FROM apps;"

done | grep -i appName

2. In the 2nd column is the title.  This next command requires you to type the title in in the exact case.  So if for example you have an application called Thunderbird you would delete it as follows;

for x in $(find /private/var/folders -name *launchpad*)

do

  sqlite3 $x/db/db "DELETE FROM apps WHERE title='Thunderbird';"

done

Replacing Thunderbird with the title you got from Step 1.

Your launchpad will now start to look cleaner.

GPG Sharing encrypted files

Occasionally you want to share a secret file, be it a password, or some credentials, or even somethings else with colleague(s) over the unsecure internet or via Email.

gpg is a useful program for achieving this but it does require you to have the public key of the person you are intending to encrypt the file for, or a shared public key to which everyone has the private one.

You ENCRYPT: gpg -z0 --output message.gpg --encrypt --recipient “Person-B" message.txt

Colleague DECRYPT: gpg --output message --decrypt message.gpg