Skip to content

Fully document #193 stream module #211

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
aboe76 merged 2 commits into from
Jan 26, 2019
Merged

Fully document #193 stream module #211

aboe76 merged 2 commits into from
Jan 26, 2019

Conversation

@noelmcloughlin
Copy link
Contributor

@noelmcloughlin noelmcloughlin commented Jan 25, 2019

This PR documents tricky nginx configuration discussed in #193.

The pillar.example is based on this example and renders config shown. Sorry for the ugly yaml.

stream {
    server {
        listen 1000;
        proxy_pass lb-1000;
    }
    server  {
        listen 53 udp;
        proxy_pass dns_servers;
    }
    server   {
        listen 12346;
        proxy_pass backend4.example.com:12346;
    }
    upstream dns_servers {

        server 192.168.136.130:53 ;
        server 192.168.136.131:53 ;
        server 192.168.136.132:53 ;
    }

    upstream lb-1000 {

        server hostname1.example.com:1000;
        server hostname2.example.com:1000;
    }
    upstream stream_backend {
        least_conn ;
        server backend1.example.com:12345 weight=5;
        server backend2.example.com:12345 max_fails=2 fail_timeout=30s ;
        server backend3.example.com:12345 max_conns=3 ;
    }
}
user www-data;
worker_processes 4;

@noelmcloughlin noelmcloughlin mentioned this pull request Jan 25, 2019
Closed
@noelmcloughlin
Copy link
Contributor Author

noelmcloughlin commented Jan 25, 2019

It might be possible to simplify few bits - suggestions welcome!!

@aboe76
Copy link
Contributor

aboe76 commented Jan 26, 2019

@noelmcloughlin I use snippets for this stuff:

nginx:
  ng:
    snippets:
      upstream_netdata_tcp:
        - upstream netdata:
          - server: 127.0.0.1:19999
          - keepalive: 64

@noelmcloughlin
Copy link
Contributor Author

noelmcloughlin commented Jan 26, 2019

Questions:
Can you model least_conn ; in snippets?
Can you have duplicate keys?
Is snippets better solution for pillar.example overall?

We should ensure the pillar.example have enough examples that yaml is not blocking them.

@aboe76
Copy link
Contributor

aboe76 commented Jan 26, 2019

@noelmcloughlin

nginx:                                                                                                       [0/604]
  ng:
    snippets:
      upstream_netdata_tcp:
        - upstream netdata:
          - server: 127.0.0.1:19999
          - keepalive: 64
      upstream_phpcgi_unix:
        - upstream phpcgi:
          - least_conn: ''
          - server: unix:/var/run/php-fpm/php-fpm-arch.sock                                                        
          - server: unix:/var/run/php-fpm/php-fpm-arch-two.sock

@noelmcloughlin
Copy link
Contributor Author

noelmcloughlin commented Jan 26, 2019

Excellent ... I'll update the PR later. thanks!!!

@aboe76
Copy link
Contributor

aboe76 commented Jan 26, 2019

@noelmcloughlin snippets are a way to split nginx configs in smaller more managable items,
which can be included in other files.

I have used this formula to mimic: https://github.com/perusio/drupal-with-nginx

@noelmcloughlin
Copy link
Contributor Author

noelmcloughlin commented Jan 26, 2019

Will duplicate server keys work in snippets?

Note: this is different use case to server: unix:/var/run/php-fpm/php-fpm-arch-two.sock

For each instance in yaml I appended 1xwhitespace
'server':
'server ':
'server ':

@aboe76
Copy link
Contributor

aboe76 commented Jan 26, 2019

@noelmcloughlin I don't know, I use pillarstack and varstack to combine my pillar data...

@aboe76
Copy link
Contributor

aboe76 commented Jan 26, 2019

But I nginx:ng can use multiple keys:

see the nginx.conf for listen

@aboe76
Copy link
Contributor

aboe76 commented Jan 26, 2019

@noelmcloughlin one other thing usefull for snippets part with multiple keys:

nginx:
  ng:
    snippets:
      cloudflare_proxy:
        - set_real_ip_from: 103.21.244.0/22
        - set_real_ip_from: 103.22.200.0/22
        - set_real_ip_from: 103.31.4.0/22
        - set_real_ip_from: 104.16.0.0/12
        - set_real_ip_from: 108.162.192.0/18
        - set_real_ip_from: 131.0.72.0/22
        - set_real_ip_from: 141.101.64.0/18
        - set_real_ip_from: 162.158.0.0/15
        - set_real_ip_from: 172.64.0.0/13
        - set_real_ip_from: 173.245.48.0/20
        - set_real_ip_from: 188.114.96.0/20
        - set_real_ip_from: 190.93.240.0/20
        - set_real_ip_from: 197.234.240.0/22
        - set_real_ip_from: 198.41.128.0/17
        - set_real_ip_from: 2400:cb00::/32
        - set_real_ip_from: 2606:4700::/32
        - set_real_ip_from: 2803:f800::/32
        - set_real_ip_from: 2405:b500::/32
        - set_real_ip_from: 2405:8100::/32
        - set_real_ip_from: 2c0f:f248::/32
        - set_real_ip_from: 2a06:98c0::/29

@aboe76
Copy link
Contributor

aboe76 commented Jan 26, 2019 •
edited
Loading

Or something truly awfull:

ngingx:
  ng:
    snippets:
      blacklist:
        - map $http_user_agent $bad_bot:
          - default: 0
          - '~*^Lynx': 0
          - '~*malicious': 1
          - '~*bot': 1
          - '~*backdoor': 1
          - '~*crawler': 1
          - '~*bandit': 1
          - libwww-perl: 1
          - '~(?i)(httrack|htmlparser|libwww)': 1

together with:

nginx:
  ng:
    servers:
      managed:
        default:
          enabled: True
          config:
            - server:
              - if ($bad_bot):
                - return: 444
              - include: /etc/nginx/snippets/blacklist

@noelmcloughlin
Copy link
Contributor Author

noelmcloughlin commented Jan 26, 2019

Hi @aboe76 please review this now....

@aboe76 aboe76 merged commit 854d1ed into saltstack-formulas:master Jan 26, 2019
@aboe76
Copy link
Contributor

aboe76 commented Jan 26, 2019

@noelmcloughlin nice work

@noelmcloughlin noelmcloughlin deleted the fix7 branch January 26, 2019 19:33
@garrettw garrettw mentioned this pull request Apr 10, 2019
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@aboe76 aboe76 aboe76 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@noelmcloughlin @aboe76