Speeding up ansible playbooks or running many ansible playbooks in parallel
Home > CentOS > CentOS 6.x > System administration tools > ansible > Ansible tips and tricks > Speeding up ansible playbooks or running many ansible playbooks in parallel
To speed up ansible playbook module named firewall was introduced a while back. Information about fireball is available at http://jpmens.net/2012/10/01/dramatically-speeding-up-ansible-runs/ However as per https://github.com/ansible/ansible/issues/10599 it was recommended to use ansible acceleration feature instead of using fireball from ansible 1.3 onwards. Further as per http://docs.ansible.com/ansible/playbooks_acceleration.html ansible 1.5 onwards there is no need to use acceleration feature. Ansible 2.0 has introduced strategy defined at http://docs.ansible.com/ansible/playbooks_strategies.html but ansible 2.0 is not available as a package in latest updated CentOS 6.x at time of this writing. Thus, other ways of speeding up ansible playbooks are discussed here.
Example sequential site
Various methods of speed-up will be discussed based on this example sequential site:
hosts
[test1] 192.168.122.102 [test2] 192.168.122.103
main.yaml
- include: test1.yaml - include: test2.yaml
test1.yaml
- name: Configure test1 machine hosts: test1 remote_user: root vars: - test1_color : "Yellow" tasks: - name: Sleep for 30 seconds shell: sleep 30 notify: - print1 - print2 - name: Echo test1_color debug: msg="color is {{test1_color}}" handlers: - name: print1 shell: echo "print1 handler executed" - name: print2 debug: msg="print2 handler executed"
test2.yaml
- name: Configure test1 machine hosts: test2 remote_user: root vars: - test2_color : "Green" tasks: - name: Echo test2_color debug: msg="color is {{test2_color}}"
Here main.yaml includes the other two host configuration playbooks which execute sequentially. Thus setup of machine test2 does not starts till 30 second sleep times out on test1
Shell script based speed-up
A shell-script based speed up is possible by using same hosts, test1.yaml and test2.yaml files. But instead of using main.yaml use:
main.sh
#!/bin/bash ansible-playbook -i hosts test1.yaml > test1.log & ansible-playbook -i hosts test2.yaml > test2.log &
In this case test2.log would have test2.yaml output well before test1.log contains output related to test1.yaml. This is same as using multiple shells and executing test1.yaml and test2.yaml in different shells parallely.
ansible main.yaml based parallelization
The running of two shell-scripts in background (as done in previous 'Shell script based speed up' section) can also be done using modified main.yaml as follows:
main.yaml
- name: Run test1 and test2 in parallel hosts: ansible remote_user: root vars: - working_dir: "/root/ansible/03-ansible" tasks: - name: Run test1.yaml file shell: ansible-playbook -i hosts test1.yaml > test1.log & args: chdir: "{{working_dir}}" register: test1_output - name: Run test2.yaml file shell: ansible-playbook -i hosts test2.yaml > test2.log & args: chdir: "/root/ansible/03-ansible" register: test2_output - name: Print test1 and test2 error output, if any debug: msg="Test1 basic status is {{test1_output}} and Test2 basic status is {{test2_output}}. For normal output refer test1.log and test2.log (Not working)"
This example was created hoping that output of both ansible-playbook commands would get captured in respective log files. Unfortunately that is not happening. The example needs more work for obtaining ansible output for troubleshooting or status check purposes. However, both scripts do execute in parallel so this achieves speed up same as before. This method also has overhead of connecting to localhost (ansible-server) over SSH.
Thus best method is to use shell script as described before
fireball based speed-up (Obsolete)
Fireball has a few dependencies that can be installed using:
one_time_setup.yaml
- name: One time setup of test1 and test2 for fireball hosts: ansible:test1:test2 remote_user: root gather_facts: false connection: ssh tasks: - yum: name={{item}} with_items: - python-pip - gcc - python-devel - gcc-c++ - python-zmq - pip: name=pyzmq state=present - pip: name=pyasn1 - pip: name=PyCrypto - pip: name=python-keyczar
After this a modified main.yaml file, such as:
- name: Start firewall on test1 and test2 hosts: ansible:test1:test2 remote_user: root tasks: - name: Start fireball action: fireball - include: test1.yaml - include: test2.yaml
should start fireball on all three nodes before starting execution of test1.yaml and test2.yaml. These two .yaml files must have modification such as:
- name: Configure test1 machine hosts: test1 connection: fireball remote_user: root
to indicate use of fireball connection.
This example was found to be Not working. Suggestions on how to make it work are welcome. Refer http://docs.ansible.com/ansible/fireball_module.html
Also refer http://stackoverflow.com/questions/21158689/ansible-deploy-on-multiple-hosts-in-the-same-time for good answer on same topic.
Home > CentOS > CentOS 6.x > System administration tools > ansible > Ansible tips and tricks > Speeding up ansible playbooks or running many ansible playbooks in parallel