You speak and write some English, but not near native quality. But you need to publish English texts, even if just as a chat message to a team. You can’t run each chat message by a native speaker friend, because they have lots of other things to do. So you are looking for machine help.
Let AI do the job. AI is said to be occasionally hallucinating about facts and conclusions, but it’s good about grammar and spelling. So you create the content and AI can adjust the words.
Someone said, “AI is like a parrot and a text mechanic.” That’s what I’m trying to use it for. Comparable to a pocket calculator in Mathematics. Not a supercomputer or Wolfram Alpha, but a pocket calculator.
Because your content may not be public, eg. intended for an internal message within your organization, you cannot use online services like Grammarly. Therefore, you must utilize a locally running AI. A colleague pointed me to GPT4All.io . Their smaller models run on computers with as little as 8 GB of RAM, while the larger models require 16 GB. You don’t need a GPU; the standard CPU of your desktop/notebook computer is sufficient. GPT4All supports all major operating systems: Linux, Mac, Windows.
Note: Do this when you have good, unmetered Internet connectivity. The models are approximately 3-8 gigabytes each to download.
If you want to experiment with other models, repeat step 6.
If you want to download only one model, choose “Mistral Instruct”.
[INST]Please correct any grammatical or spelling errors in this text
while staying as close to the original content as possible.
Do not consider the context when making your corrections
and only provide the revised version.
%1 [/INST]
To check another sentence or paragraph, just repeat from step 3.
GPT4All can help you improve your English communication by providing grammar and spelling corrections for your written text. This tool uses artificial intelligence to identify errors in your writing and suggest corrected versions. You can use this app daily as a part of your writing routine.
]]>Cryptomator-x.y.z-platform.exe
. x.y.z
is a placeholder for the version number, 1.2.3 (sic) at the time of this writing (mid-April 2017). platform
is a placeholder that changes for the 32- vs. 64-bit version. A full example name is Cryptomator-1.2.3-x86.exe for Cryptomator version 1.2.3 for 32-bit Windows. Use the version that matches the bitness of your operating system. If in doubt, use the 32-bit version.When you start Cryptomator for the first time, or any time you have no vaults set up, it shows you an emtpy list on its left and a line from a text “click here to add new vaults” to a “+” button on its bottom. The “+” button is always there, if you have any vaults defined already. Just the text and the line is visible only when you have no vaults defined in your copy of Cryptomator.
Click this “+” button. A pop down should appear offering you two buttons, so you can select whether you want to create a new vault or open an existing vault.
C:\Users\yourname\Dropbox
. In the “file name” line of the save-as dialogue, type the name of the new directory you want to use as a vault, for instance confidential-stuff
confidential-stuff
under your C:\Users\yourname\Dropbox
directory and thus could be synchronized by Dropbox. You will be returned to Cryptomator’s list of vaults and a password dialogue.C:\Users\yourname\Dropbox\confidential-stuff
. You need to locate and select the file named masterkey.cryptomator
within the vault’s directory.masterkey.cryptomator
file. This will create a new line in the list of vaults on the left of Cryptomator’s main window.Now you can use the mapped drive, e.g. X:, for your confidential data in the same way as you would use a network drive - which it really is (Cryptomator internally uses the WebDAV protocol to present the decrypted content of a vault). Whenever you save a file residing within the vault, the encrypted version gets updated within a few seconds and thus your cloud storage’s synchronization tool can immediately start synchronizing the encrypted version to the cloud.
You can have multiple vaults open at the same time, they just need to have different drive letters.
When you have completed your work on the contents of a vault, you can/should close it.
The instructions above are valid for Windows. Linux is a little different:
http://localhost:33639/ZLhq_XiwRsGq/name_of_your_vault
. You can change the port number and the name_of_your_vault in Cryptomator’s “more options” dialogue.Most likely, you have been looking into using cloud storage to synchronize, share or backup data. But you did not dare to use cloud storage, because your data is confidential and you don’t want the cloud storage provider to read it.
Using a VeraCrypt container or an encrypted ZIP file would solve the trust issues, but has disadvantages in handling:
The solution is an app that encrypts each file individually, transparent to the user.
Boxcryptor used to be known for this, but around 2012 a cryptographic flaw had been uncovered in the foundation (encFS) of their version 1. There is version 2 now, but it is relatively expensive, especially if you are using it in a business context.
In early 2016, a group of Germans (why so often them?) published Cryptomator, a free, open source app, to provide automatic, transparent, client-side file-by-file encryption of entire directories, including files, subdirectories etc.
Cryptomator is available for Windows, Mac and Linux. Beta versions are available for iOS and Android.
This is only a quick overview. For details see the project home page and the documentation. Be assured, using Cryptomator is easy. So easy, they won a special prize for Usable Security and Privacy on the CeBIT Innovation Award 2016 (“joint venture” of the German Federal Ministry of Education and Research and the large annual IT exhibition CeBIT Hannover) .
At first you download and install the app from cryptomator.org. Only use this source, as there are malware-ridden fake copies around.
Then you set up a vault (or several vaults). A vault is essentially a directory, usually located inside your cloud app’s synchronization space. Within the app, you select that directory, assign a password and an optional drive letter. If someone (maybe you on a different device) is sharing a vault with you, you can’t create it (the other guy already did that), but instead “open” it, specifying the directory from your point of view.
Example (on Windows): the encrypted data will reside in C:\Users\Yourname\Dropbox\confidential_stuff\ with its decrypted version being visible (and editable) as drive X:
You can share a vault with others by sharing access to the (cloud) storage and telling them the password of the vault. There is only one password per vault. Every user is using this same password for this vault. So if you want to share a different set of files with a different group of people, you need to create/use a different vault with a (most likely) different password.
When you want to access a vault’s data, you start the Cryptomator app, select your vault, enter the vault’s password and click on “unlock vault”. Then (obviously only if you got the password right) the decrypted version gets mapped to drive X: (or what ever you chose when setting up this vault on your computer) and a file explorer window opens, showing the vault’s decrypted content.
Then you can use drive X: as you would use a network drive. (Which in fact it is. Cryptomator internally uses WebDAV.)
You can edit, create, delete etc. files and directories to any nesting level. You can e.g. double click a text or graphics file residing within a vault to open it, using your default text or graphics viewer.
Some applications on some platforms don’t like the WebDAV network drive Cryptomator creates for the decrypted side. One example I frequently encounter is TaskCoach, my favorite self-management system. Version 1.4.3 can open a .tsk file on Cryptomator on Windows, but cannot on Linux. But TaskCoach version 1.4.2 on Linux was able to open a Cryptomator (Linux) stored tsk-file, because of different lock file handling that got “fixed” in 1.4.3.
I have not run into size limitations. One vault I’m using daily has more than 900 files in more than 300 folders, including a local Git repository of a small software development project, totaling more than 5 GB, synchronized to a NextCloud server.
Whether Cryptomator can be your tool of choice depends on your use case.
Cryptomator is great when used for the purpose it was intended: using a trusted computer to store files on untrusted (cloud) storage.
But you should not use Cryptomator if
Cryptomator is a great tool for encrypting your files on lesser trusted storage, as long as you don’t mind the locations of encrypted data being well visible. Using Cryptomator is so easy that they won a CeBIT 2016 special prize for Usable Security and Privacy.
]]>To keep this article short, I’ll only summarize the results, writing only brief details about the findings. If there is public interest in how I came to the conclusion about one or the other product, please write to me and I may write a public or private summary why I think product X might be doing things users might not want.
Extensions in this list look clean. I did not find suspicious traffic. Of course this doesn’t mean there is no unwanted traffic. I may just have failed to spot the issues…
Extensions in this list clearly send encrypted traffic beyond what is necessary for legitimate operation, or send tracking IDs (by cookie or request data) despite claiming to protect from tracking. To me, encrypted traffic within an already HTTPs encrypted connection indicates the desire to hide something. Hiding content is o.k. for password managers or chat systems, but not for products that should not send anything first place.
Do not use these products if you are visiting confidential members-only websites or if for other reasons would mind your browsing history ending up in the hands of ad-optimizers or anyone else willing to pay for web analytics of other sites.
The “Welcome to Ghostery” page sent tracking requests to analytics.cliqz.com, retrieving a piwik script and then sending a tracking message.
IMO promising to stop trackers but at the same time tracking users is not fair play.
Extensions in this list are doing things that look suspicious. They may be benign, but this is hard to tell. I do not recommend using these products with sensitive data, such as health, banking or confidential Intranet sites.
During surfing, apparently all forms encountered (field names, no values) are sent to mapping.abine.com . This submission includes the domain. So they seem to be mapping innocent web sites for form metadata.
Given the amount of trust needed for a password store, there is too much encrypted traffic compared to the purpose.
No highly suspicious traffic, although significant amounts of cookies, including optimizely.com, in the weather data requests. Calls to pixel.rubiconproject.com including ruid cookies, which probably enable user tracking.
Most browsers make use of one of these systems to help protecting you from social engineering attacks and malicious downloads. Each system makes the browser send browsing information to a major pool, which may not be what you want, privacy-wise. You need to make a judgment whether you want to give up some privacy in return for some protection against malicious websites.
Turned on by default in Mozilla Firefox.
… works by checking the sites that you visit against lists of reported phishing, unwanted software and malware sites.
…
When you download an application file, Firefox checks the site hosting it against a list of sites known to contain “malware”. If the site is found on that list, Firefox blocks the file immediately, otherwise it asks Google’s Safe Browsing service if the software is safe by sending it some of the download’s metadata … … including the name, origin, size and a cryptographic hash of the contents.
This means, essentially all downloaded executable files are reported to Google. If you don’t want that, and don’t need the protection, turn it off by going to Firefox > Preferences > Security > “Block dangerous and deceptive content” . Do work from bottom to top, as otherwise some boxes stay ticked but greyed out.
According to Wikipedia, every website and download is checked against a local list of popular legitimate websites; if the site is not listed, the entire address is sent to Microsoft for further checks. So all visited URLs of confidential Intranet sites, certainly not in the list of popular legitimate websites, will get sent to Microsoft.
You are paying this privacy price in exchange for a high rate (95-99% according to Wikipedia) of protection against social engineered malicious websites. So you need to “choose your poison”.
]]>Argument | Effect |
---|---|
--help |
Show this help message and exit. |
--anticache |
Strip out request headers that might cause the server to return 304-not-modified. |
-r RFILE |
Read flows from file. |
-w STREAMFILE |
Write flows to file. |
-a STREAMFILE |
Append flows to file. |
--anticomp |
Try to convince servers to send us un-compressed data. |
-n |
Don’t start a proxy server. Useful for offline analyzing a previously captured stream. |
-f FILTER |
Filter view expression. |
Key | Function |
---|---|
? |
show context sensitive help (very useful) |
q |
quit / return to previous page |
O |
Options |
Key | Function |
---|---|
g , G |
go to beginning, end |
pg up /down or space |
page up/down |
arrows | up, down, left, right |
Key | Function |
---|---|
enter |
view flow (go to Flow View) |
e |
toggle eventlog |
tab |
tab between eventlog and flow list |
o |
set flow order, will ask for sort criteria - useful for finding and deleting large download flows before saving stream |
d |
delete flow |
f |
filter view |
w |
save flows |
z |
zap (clear) flow list or eventlog |
q |
quit mitmproxy |
Key | Effect |
---|---|
tab |
next tab (cycles through Request - Response - Detail) |
w |
save all flows matching current view filter |
f |
load full body data |
space |
next flow |
/ |
search (case sensitive) |
n |
repeat search forward |
N |
repeat search backwards |
q |
return to flow list |
Expression | Description |
---|---|
~b regex |
Body |
~c int |
HTTP response code |
~d regex |
Domain |
~e |
Match error |
~h regex |
Header |
~hq regex |
Request header |
~m regex |
Method |
~q |
Match request with no response |
~s |
Match response |
~t regex |
Content-type header |
~u regex |
URL |
! |
unary not |
& |
and |
\| |
or |
(...) |
grouping |
URLs containing “lastpass”: lastpass
Exclude flows connecting to google and doubleclick: (! google) & (! doubleclick)
All requests sending cookies: ~hq Cookie
This is essentially the output of mitmproxy --help
, formatted for better web page reading.
usage: mitmproxy [options]
Argument | Effect |
---|---|
-h, --help |
show this help message and exit |
--conf PATH |
Configuration file |
--version |
show program’s version number and exit |
--sysinfo , --shortversion |
show program’s short version number and exit |
--anticache |
Strip out request headers that might cause the server to return 304-not-modified |
--cadir CADIR |
Location of the default mitmproxy CA files. (~/.mitmproxy ) |
--host |
Use the Host header to construct URLs for display |
-q , --quiet |
Quiet |
-r RFILE , --read-flows RFILE |
Read flows from file. |
-s "script.py --bar" , --script "script.py --bar" |
Run a script. Surround with quotes to pass script arguments. Can be passed multiple times. |
-t FILTER , --stickycookie FILTER |
Set sticky cookie filter. Matched against requests. |
-u FILTER , --stickyauth FILTER |
Set sticky auth filter. Matched against requests. |
-v , --verbose |
Increase log verbosity. |
-w STREAMFILE , --wfile STREAMFILE |
Write flows to file. |
-a STREAMFILE , --afile STREAMFILE |
Append flows to file. |
-z , --anticomp |
Try to convince servers to send us un-compressed data. |
-Z SIZE , --body-size-limit SIZE |
Byte size limit of HTTP request and response bodies. Understands k/m/g suffixes, i.e. 3m for 3 megabytes. |
--stream SIZE |
Stream data to the client if response body exceeds the given threshold. If streamed, the body will not be stored in any way. Understands k/m/g suffixes, i.e. 3m for 3 megabytes. |
--upstream-auth UPSTREAM_AUTH |
Add HTTP Basic authentcation to upstream proxy and reverse proxy requests. Format: username:password |
--palette {dark,light,lowdark,lowlight,solarized_dark,solarized_light} |
Select color palette: lowlight, light, lowdark, solarized_light, solarized_dark, dark |
--palette-transparent |
Set transparent background for palette. |
-e , --eventlog |
Show event log. |
--follow |
Focus follows new flows. |
--order {time,method,url,size} |
Flow sort order. |
--no-mouse |
Disable mouse interaction. |
Argument | Effect |
---|---|
-R REVERSE_PROXY , --reverse REVERSE_PROXY |
Forward all requests to upstream HTTP server: http[s]://host[:port] . Clients can always connect both via HTTPS and HTTP, the connection to the server is determined by the specified scheme. |
--socks |
Set SOCKS5 proxy mode. |
-T , --transparent |
Set transparent proxy mode. |
-U UPSTREAM_PROXY , --upstream UPSTREAM_PROXY |
Forward all requests to upstream proxy server: http://host[:port] |
Option | Effect |
---|---|
-b ADDR , --bind-address ADDR |
Address to bind proxy to (defaults to all interfaces) |
-I HOST , --ignore HOST |
Ignore host and forward all traffic without processing it. In transparent mode, it is recommended to use an IP address (range), not the hostname. In regular mode, only SSL traffic is ignored and the hostname should be used. The supplied value is interpreted as a regular expression and matched on the ip or the hostname. Can be passed multiple times. |
--tcp HOST |
Generic TCP SSL proxy mode for all hosts that match the pattern. Similar to --ignore , but SSL connections are intercepted. The communication contents are printed to the log in verbose mode. |
-n , --no-server |
Don’t start a proxy server. Useful for offline analyzing a previously captured stream. |
-p PORT , --port PORT |
Proxy service port. Default: 8080 |
--http2 , --no-http2 |
Explicitly enable/disable HTTP/2 support. Disabled by default until major websites implement the spec correctly. Default value will change in a future version. |
--no-websocket , --websocket |
Explicitly enable/disable WebSocket support. Enabled by default. |
--raw-tcp , --no-raw-tcp |
Explicitly enable/disable experimental raw tcp support. Disabled by default. Default value will change in a future version. |
--spoof-source-address |
Use the client’s IP for server-side connections. Combine with –upstream-bind-address to spoof a fixed source address. |
--upstream-bind-address UPSTREAM_BIND_ADDRESS |
Address to bind upstream requests to (defaults to none) |
Option | Effect |
---|---|
--cert SPEC |
Add an SSL certificate. SPEC is of the form [domain=]path . The domain may include a wildcard, and is equal to * if not specified. The file at path is a certificate in PEM format. If a private key is included in the PEM, it is used, else the default key in the conf dir is used. The PEM file should contain the full certificate chain, with the leaf certificate as the first entry. Can be passed multiple times. |
--ciphers-client CIPHERS_CLIENT |
Set supported ciphers for client connections. (OpenSSL Syntax) |
--ciphers-server CIPHERS_SERVER |
Set supported ciphers for server connections. (OpenSSL Syntax) |
--client-certs CLIENTCERTS |
Client certificate file or directory. |
--no-upstream-cert |
Don’t connect to upstream server to look up certificate details. |
--add-upstream-certs-to-client-chain |
Add all certificates of the upstream server to the certificate chain that will be served to the proxy client, as extras. |
--insecure |
Do not verify upstream server SSL/TLS certificates. |
--upstream-trusted-cadir SSL_VERIFY_UPSTREAM_TRUSTED_CADIR |
Path to a directory of trusted CA certificates for upstream server verification prepared using the c_rehash tool. |
--upstream-trusted-ca SSL_VERIFY_UPSTREAM_TRUSTED_CA |
Path to a PEM formatted trusted CA certificate. |
--ssl-version-client {SSLv3,all,TLSv1_1,TLSv1_2,secure,SSLv2,TLSv1} |
Set supported SSL/TLS versions for client connections. SSLv2, SSLv3 and ‘all’ are INSECURE. Defaults to secure, which is TLS1.0+. |
--ssl-version-server {SSLv3,all,TLSv1_1,TLSv1_2,secure,SSLv2,TLSv1} |
Set supported SSL/TLS versions for server connections. SSLv2, SSLv3 and ‘all’ are INSECURE. Defaults to secure, which is TLS1.0+. |
Option | Effect |
---|---|
--noapp |
Disable the mitmproxy onboarding app. |
--app-host APP_HOST |
Domain to serve the onboarding app from. For transparent mode, use an IP when a DNS entry for the app domain is not present. Default: mitm.it |
--app-port 80 |
Port to serve the onboarding app from. Does not need to be reachable from outside, because the client will go through the built-in proxy anyway. |
Option | Effect |
---|---|
-c PATH , --client-replay PATH |
Replay client requests from a saved file. |
Option | Effect |
---|---|
-S PATH , --server-replay PATH |
Replay server responses from a saved file. |
-k , --replay-kill-extra |
Kill extra requests during replay. |
--server-replay-use-header SERVER_REPLAY_USE_HEADERS |
Request headers to be considered during replay. Can be passed multiple times. |
--norefresh |
Disable response refresh, which updates times in cookies and headers for replayed responses. |
--no-pop |
Disable response pop from response flow. This makes it possible to replay same response multiple times. |
--replay-ignore-content |
Ignore request’s content while searching for a saved flow to replay |
--replay-ignore-payload-param SERVER_REPLAY_IGNORE_PAYLOAD_PARAMS |
Request’s payload parameters (application/x-www-form-urlencoded or multipart/form-data) to be ignored while searching for a saved flow to replay. Can be passed multiple times. |
--replay-ignore-param SERVER_REPLAY_IGNORE_PARAMS |
Request’s parameters to be ignored while searching for a saved flow to replay. Can be passed multiple times. |
--replay-ignore-host |
Ignore request’s destination host while searching for a saved flow to replay |
Replacements are of the form “/pattern/regex/replacement”, where the separator can be any character. Please see the documentation for more information.
Option | Effect |
---|---|
--replace PATTERN |
Replacement pattern. |
--replace-from-file PATH |
Replacement pattern, where the replacement clause is a path to a file. |
Header specifications are of the form “/pattern/header/value”, where the separator can be any character. Please see the documentation for more information.
Option | Effect |
---|---|
--setheader PATTERN |
Header set pattern. |
Specify which users are allowed to access the proxy and the method used for authenticating them.
Option | Effect |
---|---|
--nonanonymous |
Allow access to any user long as a credentials are specified. |
--singleuser USER |
Allows access to a a single user, specified in the form username:password . |
--htpasswd PATH |
Allow access to users specified in an Apache htpasswd file. |
See help in mitmproxy for filter expression syntax.
Option | Effect |
---|---|
-i INTERCEPT , --intercept INTERCEPT |
Intercept filter expression. “Intercepting” means waiting for user approval/modification before passing on/back to server/client. |
-f FILTER , --filter FILTER |
Filter view expression. |
This is essentially a copy of the help screens of mitmproxy, as retrieved by the ?
key.
i set interception pattern
O options
q quit / return to previous page
Q quit without confirm prompt
R replay of requests/responses from file
j, k down, up
h, l left, right (in some contexts)
g, G go to beginning, end
space page down
pg up/down page up/down
ctrl+b/ctrl+f page up/down
arrows up, down, left, right
A accept all intercepted flows
a accept this intercepted flow
b save request/response body
C export flow to clipboard
d delete flow
D duplicate flow
e toggle eventlog
E export flow to file
f filter view
F toggle follow flow list
L load saved flows
m toggle flow mark
M toggle marked flow view
n create a new request
o set flow order
r replay request
S server replay request/s
U unmark all marked flows
v reverse flow order
V revert changes to request
w save flows
W stream flows to file
X kill and delete flow, even if it's mid-intercept
z clear flow list or eventlog
tab tab between eventlog and flow list
enter view flow
| run script on this flow
A accept all intercepted flows
a accept this intercepted flow
b save request/response body
C export flow to clipboard
D duplicate flow
d delete flow
e edit request/response
f load full body data
m change body display mode for this entity
(default mode can be changed in the options)
*a*utomatic: automatic detection
h*e*x: Hex
*h*tml: HTML
*i*mage: Image
*j*avascript: JavaScript
j*s*on: JSON
*u*rlencoded: URL-encoded data
*r*aw: raw data
*x*ml: XML
E export flow to file
r replay request
V revert changes to request
v view body in external viewer
w save all flows matching current view filter
W save this flow
x delete body
z encode/decode a request/response
tab next tab (cycles through Request - Response - Detail)
h, l previous tab, next tab
space next flow
| run script on this flow
/ search (case sensitive)
n repeat search forward
N repeat search backwards
This is essentially a copy of http://docs.mitmproxy.org/en/stable/features/filters.html .
Many commands in mitmproxy and mitmdump take a filter expression. Filter expressions consist of the following operators:
Expression | Description |
---|---|
~a |
Match asset in response: CSS, Javascript, Flash, images. |
~b regex |
Body |
~bq regex |
Request body |
~bs regex |
Response body |
~c int |
HTTP response code |
~d regex |
Domain |
~dst regex |
Match destination address |
~e |
Match error |
~h regex |
Header |
~hq regex |
Request header |
~hs regex |
Response header |
~http |
Match HTTP flows |
~m regex |
Method |
~marked |
Match marked flows |
~q |
Match request with no response |
~s |
Match response |
~src regex |
Match source address |
~t regex |
Content-type header |
~tcp |
Match TCP flows |
~tq regex |
Request Content-Type header |
~ts regex |
Response Content-Type header |
~u regex |
URL |
! |
unary not |
& |
and |
\| |
or |
(...) |
grouping |
~h
, ~hq
, ~hs
) is against a string of the form “name: value”.&
.URL containing “google.com”: google\.com
Requests whose body contains the string “test”: ~q ~b test
Anything but requests with a text/html content type: !(~q & ~t "text/html")
Usually, at least in the German market, all of this is done by the web hosting company. But recently I had the task to perform this not for a German, but for a Kyrgyz customer. Things do work different there. European hosting companies, if they offer .kg-domains at all, charge 180 to 270 USD per year just for the domain registration, while domain.kg does it for the equivalent of 37 USD per year. So I set out to do it on my own.
For reliability, all Internet domain names are required to be resolved by at least two different name servers. Make sure you have access to both of them. If not, get a small VPS (Virtual Private Server) from a provider like Hetzner or HostEurope. Details of getting a VPS are beyond the scope of this text.
In several places you need to enter numeric IP-Addresses. In this example I’m using ns2.example.net (9.8.7.6) for the primary (authoritative) name server and ns1.example.net (1.2.3.4) for the secondary name server.
Through an appropriate registrar, domain.kg in this case, “buy the domain”. You will need to specify the DNS names and IP addresses of the name servers. In the case I worked with, the customer did this on his own from within Kyrgyzstan.
Example IP-Address: 9.8.7.6
The server selected to become the primary name server is a VPS running Debian 7 Linux.
apt-get install bind9
zone "example.kg" {
type master;
file "/etc/bind/example.kg.hosts";
also-notify { 1.2.3.4; };
};
$ttl 38400
$origin example.kg.
@ SOA ns2.example.net. hostmaster.example.net. (
2017020300
21600
3600
604800
86400 )
@ NS ns2.example.net.
@ NS ns1.example.net.
@ A 9.8.7.6
www A 9.8.7.6
Example IP-Address: 1.2.3.4
apt-get install bind9
zone "example.kg" {
type slave;
file "/var/cache/bind/example.kg.hosts";
masters { 9.8.7.6; };
};
The tool of choice on Linux seems to be zonecheck (man page).
apt-get install zonecheck
zonecheck --ns ns2.example.net\;ns1.example.net example.kg
The backslash before the semicolon is important to avoid the shell misinterpreting the semicolon as and of command.Assuming apache2 is already installed.
adduser example
cd /home/example
mkdir public_html
chown example.www-data public_html
chmod 755 public_html
mkdir logs
chown example.www-data public_html
chmod 775 logs
This enables the webmaster to SFTP (FTP over SSH) into the web server and upload content. No FTP server software needed if you can SSH into the web server.
Create a dummy index.html web page and upload it to /home/example/public_html by FileZilla or equivalent, using the SFTP (FTP over SSH) protocol.
apt-get install apache2
/etc/apache2/sites-available
, copy 000-default.conf
to example.conf
and edit to suit your needs. Important directives include: ServerName www.example.kg
ServerAdmin webmaster@example.kg
DocumentRoot /home/example/public_html
<Directory /home/example/public_html/ >
Require all granted
</Directory>
ErrorLog /home/example/logs/error.log
CustomLog /home/example/logs/access.log combined
a2ensite example
Wait until the DNS entry has publicly spread.
Follow the instructions on https://certbot.eff.org/#debianjessie-apache
This creates and enables a new site example-le-ssl.conf in the Apache configuration.
Yes, letsencrypt is really that easy, if you are using a supported OS/web server pair.
This is the hardest part. I consider it out of reach of a less-than-full-time administrator to manage an SMTP server with reasonable spam filtering.
Options include:
In this case we decided to go with df.eu, as about a dozen mailboxes were needed.
DF’s hosting package turned out to be all we need, so I changed all the DNS records to point to df’s servers, let the customer change the domain’s DNS server to DF’s - and considered doing the steps described here as an exercise that won’t go into long term production.
]]>Mitmproxy’s home page is https://mitmproxy.org/ . It is a suite consisting of the interactive program Mitmproxy and the stream dumper (think tcpdump) mitmdump . The name is a combination of the acronym Man In The Middle (the type of attack performed here) and proxy, a type of server widely use especially in company’s internal networks.
You absolutely need a 64-bit OS to run Mitmproxy on. Attempting it on a 32-bit system fails with odd error messages you won’t at first glance relate to 64-vs-32-bit issues.
Mitmproxy is at home on Linux, but there is also a Windows version available. I did it on a desktop PC running Ubuntu Mate 16.04 LTS, 64-bit.
Working along http://docs.mitmproxy.org/en/latest/install.html :
mitmproxy-1.0.2-linux.tar.gz
from https://github.com/mitmproxy/mitmproxy/releases .tar xvzf mitmproxy-1.0.2-linux.tar.gz
results in three binaries: mitmdump
, mitmproxy
and mitmweb
../mitmdump --help
shows a long list of options, I won’t repeat here. I found needing only very few of them../mitmproxy
. This is an interactive program (character based, curses-like user interface), initially showing one line per flow (HTTP request-response pair).192.168.127.44:8080
as a proxy for all protocols. In my first test this was Firefox, and it could well have been on a different computer, as long as it can talk TCP on port 8080 to the Mitmproxy machine. 192.168.127.44 happened to be the IP address of the Mitmproxy machine in my home network at the time of testing.test.stream
../mitmproxy -w test2.stream
--anticache
(modify the request to make the server re-send content possibly cached by the browser) and --anticomp
(prevent the server from sending compressed data) options can be very relevant, so my recommended command line is ./mitmdump --anticache --anticomp -a log.stream
. -a
means append instead of -w
for write/overwrite. I put this line into a little shell script /home/martin/bin/catch-as-catch-can
, to be able to start catching without having to remember all these options.e.g. ./mitmdump -n -r infile -w outfile "~m post"
reads infile
and creates outfile
, containing only streams using the post
method (~m
) .
For a full list of command line options see http://docs.mitmproxy.org/en/latest/mitmdump.html
Filter expressions are described in detail in http://docs.mitmproxy.org/en/latest/features/filters.html
You can also do filtering interactively by calling ./mitmproxy -n -r infile
and then using keyboard commands:
As a testing environment, I’m using a Windows 10 preview VM inside VirtualBox.
In the Windows VM, I have set the proxy to 192.168.127.44:8080
, both in Internet settings and in admin cmd > netsh winhttp import proxy source=ie
Installing Mitmproxy’s certificate the standard way produced no error message, but both the Edge browser and Windows Update kept complaining about certificate issues.
What helped was the hint from http://docs.mitmproxy.org/en/stable/certinstall.html , Windows (automated) section: certutil.exe -importpfx Root mitmproxy-ca-cert.p12
. It asked for a “PFX Password”, but Just hitting enter, specifying an empty password, made it go through without error message.
After closing and reopening Edge I could call HTTPS websites which were happily displayed (and logged by Mitmproxy).
When starting Windows Update, Mitmproxy throws certificate verification errors.
Also the Windows 10 telemetry seems to do some non-standard things. Mitmproxy complains
192.168.127.28:54054: CONNECT watson.telemetry.microsoft.com:443
<< HTTP protocol error in client request: Unexpected EOF
After performing the steps above, I have an environment capable of logging and analyzing the data flow of apps and browser add-ons, even if they encrypt network traffic by HTTPS. The observation results of certain products are likely to be the topic of more blog posts.
]]>You might think that it is not a great deal if one external site knows of the existence of one internal page. But what if the collected referrer data of many large sites (think Dropbox and Facebook or a major newspaper) would be aggregated and sold to anyone willing to pay for it? Any interested party could get quite a large data set containing quite a detailed map about your intranet. You don’t want this, because you do place your information on an Intranet for a reason.
The news (maybe not so new to some interested groups) is, that this sale of data does happen. Companies like SimilarWeb do offer a lot of insight into the structure and visitors of websites. This includes where visitors are coming from, i.e. what site/page does link to the site being analyzed.
https://www.similarweb.com/ourdata indicates that they are getting (probably buying) data sets from several sources, Quoting from their web site:
The “panel of monitored devices” warrants a story by itself. Just look at news stories like this about the WebOfTrust browser extension. You basically need to assume that many user’s browsing histories are being collected and sold. Maybe I’m going to publish something about this issue in the near future.
The “direct measurement sources from websites” is what I’m caring about here.
As an example, I entered “ibm.com” into SimilarWeb’s search field. One of the results was “login.ng.bluemix.net” being one of their major referral sources. The domain name “login.(anything)” makes me think that this is a customers-only site. Nobody should know internal details of this. But still, when entering “login.ng.bluemix.net” into SimilarWeb’s search field, the topics tag cloud shows what it is about.
Maybe IBM is fine with the public knowing what login.ng.bluemix.net is about, but perhaps you would mind if the public knew this amount of details about your Intranet.
To verify the issue, I set up an mitmproxy in my home network and told my Firefox browser to use this proxy (full details would fill another blog post). (See addendum at the end of the post for a less technical method.) This enables me to see all HTTP(S) traffic of my browser.
At first I visited my Intranet start page http://www.home.stut.de , a static page only loading an additional local CSS file. The GET request for the CSS did contain the base address as Referer. This is no issue, since it stays local.
Then I visited http://www.home.stut.de/wordpress/ which is an almost emtpy test site, just to be able to get a feel for that platform. After GETting a bunch of internal files, Wordpress also fetched a Google Font and a Gravatar picture. This is where it got slightly scary: My browser did send the referrer http://www.home.stut.de/wordpress/
to Google and Gravatar. So Gravatar and Google now do know, and probably will tell SimilarWeb, that there is http://www.home.stut.de/wordpress/ .
Then I clicked on the link to the “Hello World” post, causing another GET from Google Fonts and Gravatar, this time with a referrer of http://www.home.stut.de/wordpress/hello-world/
.
Clicking on Categories / Uncategorized also sent a referrer to Google and Gravatar, this time http://www.home.stut.de/wordpress/category/uncategorized/
.
So my Intranet gets mapped by Google and Gravatar whenever I click on any internal link, because any internal page build generates a call to an external site. To me this is scary. This is an Intranet, not a public freebie blog platform.
In a nutshell: Stop visitor’s browsers from telling the referral URL to link target sites.
The World Wide Web Consortium is aware of the issue and working on a W3C Candidate Recommendation Referrer Policy major browsers are already implementing.
What you probably want is a Referrer Policy of “no-referrer” (never ever send a Referer HTTP header) or possibly “same-origin” (send the Referer header to your own site only, but no header to other sites) in case you need to track link flows within your Intranet. Some sources, e.g. https://html.spec.whatwg.org/multipage/semantics.html#meta-referrer , indicate that there is a legacy value of “never”, predating the definition of “no-referrer”.
Here are general alternatives of ways to deliver the referrer-policy to browsers, excerpted from §4 of W3C’s Candidate Recommendation Referrer Policy:
Referrer-Policy: no-referrer
, for each page delivered.<meta name="referrer" content="no-referrer">
(Syntax taken from https://blog.mozilla.org/security/2015/01/21/meta-referrer/ )<a href="http://example.com" referrerpolicy="no-referrer">
The official documentation is at http://httpd.apache.org/docs/current/mod/mod_headers.html
Before you start, you need to ensure mod_headers is enabled. On a Debian system, this command should do it:
a2enmod headers
In an appropriate configuration or .htaccess file, you need to place this directive:
Header set Referrer-Policy: no-referrer
On my Debian home server it did work this way:
/etc/apache2/conf-available/no-referrer.conf
a2enconf no-referrer
Placing the Header directive into a .htaccess file in the document root did have an effect only on the static pages inside the document root, but it did not have an effect on the /wordpress subdirectory. Wordpress still sent the internal referrer information to Gravatar and Google fonts. Only the server wide configuration file really did the trick.
Between the <head>
and </head>
tags of your web page or page template, insert this line:
<meta name="referrer" content="no-referrer">
You’ll need to do this on every single page that might contain links. This includes pages referring to web fonts, css files, JavaScript frameworks etc., so it can be a lot of effort - unless you have a template/content management system where you can do it in one place for all pages.
The alternative of adding a header line in a .htaccess file is better than having to do it in each page, but .htaccess files don’t always catch everything (see above).
Assume that your Intranet’s layout is probably already known to Google, SimilarWeb and their customers.
If this is not what you want, then you need to modify your web server’s configuration to ask your visitor’s browsers to not send Referer headers. Methods softer than modifying the Apache configuration most likely won’t catch everything.
A deep Linux admin friend wrote this response to this blog post:
]]>If you’re not technical enough to use a mitmproxy, you can also use the developer tools in most browsers.
NextCloud GmbH get its revenue from support contracts, starting at 1900 Euros per year, thus being only relevant for installations approaching or exeeding 50 users. Documentation on their web site is complete and concise enough to keep me from taking notes how to set up NextCloud.
The NextCloud server is essentially a set of PHP scripts that integrates into a web server (Apache preferred) running on Linux. It provides a web GUI for human users, a WebDAV interface to files, a CalDAV interface to calendars and tasks and a CardDAV interface to contact lists a.k.a. address books.
For the database backend you essentially have a choice of SQLite and MySQL (PostgreSQL is supported too, Oracle is only available for paying support contract holders). For anything larger than a single user or testing instance, MySQL is highly recommended.
Of course you need an adequate amount of file storage for all the data your users will be storing, including previous versions and deleted files. During setup, you can chose the storage path. Multiple storage backends other than a file system path are supported by “external storage” functionality. I have not tried much with these.
Installation and updates work as documented in the Admin manual, so I won’t repeat that here. The built in updater works quite well, if you loosen permissions for updating and tighten them after updating. For details see the Upgrading Nextcloud with the Nextcloud App section of the Admin manual.
This is the core of NextCloud’s capability. Every user has his own root folder which usually contains many folders with subfolders etc. It is recommended to store documents in folders or subfolders, not on the root level.
Each folder has one user as the owner. Beware: If the owner account is deleted, e.g. when the account holder leaves the organization, all files and folders inside this folder are deleted too. This means that it is very useful to create important shared folders e.g. as admin or as a special folder holding account, to avoid unwanted deletion when some key user leaves the organization.
Client file synchronization software synchronizes selected server folders with a folder on your client computer, so you have your server files always available on your computer for using them at local disk speed, even when your Internet connection is not active - think Dropbox. The desktop client automatically synchronizes all changes in both directions as soon as connectivity returns. If conflicts arise, typically because another user has modified the same file during the same time you did, both versions are kept, but one of them gets a conflict label added to the file name, similar to Dropbox.
The desktop client is available for all major platforms:
Mobile clients are avaible for
The Android client needs quite a bit of manual nudging to re-synchronize files. Maybe my use case was not anticipated by the developers: keeping a handfull of text files on the smartphone, editing them locally (with any plain text editor, my favourite is Jota Editor). If there is connectivity, in most cases the Android NextCloud client sends the file to the server immediately after I click “save”. But when there is no connectivity (I’m frequently behind a guest network that requires captive portal registration every two hours), or when I want to check for server-side changes, I need to enter the text file’s directory in the NextCloud client to manually trigger synchronisation. The Android NextCloud client does have a “synchronize now” button, but when I click it, it takes 45 minutes (sic) to complete a sync run. I don’t want to wait that long to transfer my desktop edits to the smartphone.
All client versions have multi-account capability, which is essential e.g. if you have accounts on your home NextCloud, an office NextCloud, a church nextcloud etc. Dropbox doesn’t need this, because there is only one Dropbox server. NextCloud needs multi account capability, because anyone can host his own NextCloud server.
A user can share each folder to multiple other users and/or groups. Each share, different for each user or group, can have zero or more of these permission bits set:
Setting no bit (checkmark) is equivalent to read-only sharing.
Important: The users, both sharing and receiving, of a share will see the shared folder in the root of their own directory, even if it originally was in some subfolder of the sharing user. Thus it is recommended to share only root directory folders.
Each file or folder, no matter how deep inside the folder hierarchy, can be shared by link, optionally with
This link can be sent to anyone, especially if they don’t have an account on this NextCloud.
Limitation: each file or folder can have only one (1) sharing link. If you happen to need different levels of access, consider giving an account to those who need higher levels of access, e.g. write permission.
You can edit plain text files within the web interface, which can be useful if you want to modify a piece of text while away from a cloud-synced computer, e.g if using your smartphone feels too clumsy for the amount of editing you are planning but you have a desktop browser available (I often am in this situation during customer visits).
The optional Documents app, which needs to be enabled by an administrator, can perform online editing of rich text and save it in OpenDocument .odt format, known from OpenOffice/LibreOffice.
To use calendar and tasks, an administrator needs to enable the calendar and optionally tasks apps. This is only a few mouse clicks.
Every user has a standard personal calendar and can create more calendars. Each calendar can be shared to other users and/or groups with or without write permission. That’s it. Simple but still powerful. A lot easier than other groupware systems I have managed.
Each calendar is being assigned one colour from a choice of 8 predefined colours. If you receive a calendar share, you’ll see that calendar in the same colour the owner has set for it. There is no way to change the colour other than asking the owner to do it. In the web interface, all calendars are merged into a single view, where the colours are the only way to quickly see which calendar an appointment is stored in. But even in slightly larger organizations, well exceeding 8 calendars/colours, you can practically use the web interface, because in NextCloud (in contrast to OwnCloud) you can turn off and on individual calendars.
But the web interface is not often used, because the calendars are accessible through the standard CalDAV protocol, essentially specially formatted content inside HTTP(S). Any client programs that speak a sufficiently similar dialect of CalDAV can synchronize with NextCloud.
The classic client on the desktop is Lightning, part of the widely used email client Thunderbird, available for Linux, Mac and Windows.
A cool new (to me) client is Outlook CalDAV Synchronizer. It installs a plugin to Outlook that is relatively easy to set up and synchronizes calendars, tasks and contacts. NextCloud is in their list of fully supported server types. To me this is one of the more important discoveries of software in years, because it opens a working path to group calendaring for Outlook with a much smaller footprint than a full blown Microsoft Exchange server and without the privacy issues of Microsoft Office 365.
On Android, you need a paid app. I have tried several of them. Many do not not work reliably, but CalDAV Sync by Marten Gajda (2.69 € on Google Play) is one of the few that really work. Stay away from free synchronizers as they don’t support all relevant fields or limit the number of appointments to synchronize, which can be painful if the excess gets wiped from your server.
iOS can connect natively.
If the server is set up correctly (two lines in the root level .htaccess), you can set up the mobile clients by just entering cloudserver.example.com, without needing to precisely type suffixes like /nextcloud/remote.php/dav/calendars/joe/personal
The NextCloud tasks app adds a task list to each calendar. There are many attributes to a task, even subtasks, but only some attributes get synchronized via CalDAV Sync, largely depending on the client used. I had tasks with a start date and a due date appear as multi-day calendar entries - it does look odd when you get a multi-day appointment for a little task that doesn’t matter if you do it on Monday, Tuesday or Wednesday. So you might want to restrict your use of the tasks functionality (e.g. “never use start dates”) or set aside a separate calendar for those tasks.
The web interface supports all features, while each client supports a different subset. You can use the same clients as for calendars.
NextCloud can manage contact lists, a.k.a. address books, and cater for client programs using the CardDAV protocol. I do have repeated success with these client programs:
All in all, with Nextcloud you get a privacy respecting, self hosted file synchronization service with the additional benefit of simple but reliable calendar, tasks and contact management. I recommend any small to medium organisation to consider running one when evaluating file synchronization and/or group calendaring servers.
]]>Everywhere hackers are lurking to take over web sites that happen to run on vulnerable content management systems (CMS). There are even specialized search engines helping them to find their few thousand potential victims among the millions of all websites.
The event that showed me how bad things really are, was “Drupalgeddon” in 2014. On October 15 at 4:02pm UTC (just after 5 p.m. in Central Europe, potentially just after the good guys closed their e-mail clients for the day), the Drupal Security Team published SA-CORE-2014-005 - Drupal core - SQL injection:
“A vulnerability in this [database abstraction] API allows an attacker to send specially crafted requests resulting in arbitrary SQL execution. Depending on the content of the requests this can lead to privilege escalation, arbitrary PHP execution, or other attacks. This vulnerability can be exploited by anonymous users.”
Because a patch was published, the bad guys immediately knew how to exploit this security hole. With Drupal being a widely used content management system, the hackers saw a huge prey, so within a few hours they developed an exploit and attacked a huge number of websites.
On Oct 29th, two weeks after the vulnerability and the patch was announced, the Drupal Security Team published Drupal Core - Highly Critical - Public Service announcement - PSA-2014-003, containing a sentence that, in my view, redefined the bar for web maintainers: “You should proceed under the assumption that every Drupal 7 website was compromised unless updated or patched before Oct 15th, 11pm UTC, that is 7 hours after the announcement.” (emphasis mine). So, unless you had updated your website before midnight (Central Europe), you had to assume it was hacked.
This means that, if you are responsible for the security of a website driven by a dynamic content management system, you must provide for responding (patching the CMS) to a sufficiently important announcement within less than 7 hours, no matter what the local time in your part of the world is.
In 20 years of website building, I have seen too many sites hacked to ignore this issue.
If you absolutely need a dynamic CMS, running executable code on the pulic web server, and you don’t have the ressources to do timely patching 24/7, someone else needs to do it.
What is the problem with those CMS that execute code on the public web server? The server has executable code, that can be called by literally anyone in the world, just by requesting a URL. If this code contains a security hole, e.g. by doing Bad Things when maltreated with sufficiently strange parameters, then the website has lost.
But unless you want to hand-edit all HTML pages, you do need some code to generate the data that will be sent to the visitor’s browser. An effective solution is to keep that code away from the public web server, running it on a separate development machine. This is the idea of static CMS, also known as static site generators.
The security of such websites still relies on FTP etc. being hardened, so hackers can’t mess with your static files, but this easier than securing a CMS. Usually this is your shared web-hoster’s job, who should be knowing what he is doing (not all do, unfortunately).
=> no executable code on the web server, only on the development machine.
These are essentially the limitations of a static-only website, no matter how the static files are generated.
Product website: https://jekyllrb.com/
Given that broad support, I decided to create my small business website with Jekyll instead of my 1998 homegrown generator based on M4 and GAWK.
Copied from the Quick-start Instructions on https://jekyllrb.com/docs/quickstart/
Do this on what could become your web site development machine.
~ $ gem install jekyll bundler
~ $ jekyll new myblog
~ $ cd myblog
~/myblog $ bundle exec jekyll serve
This is a quick rundown. For details see https://jekyllrb.com/docs/structure/ .
{
{content}}
and files from _includes. Can contain Liquid commands and variables, including {
%include filename%}
. Usually you need very few layouts, mostly just default.html and post.html .{
%include filename%}
directive. Can contain Liquid commands and variables, including {
%include filename%}
.Most other files and directories (all except files starting with . and directories starting with _) are copied to _site upon build. Whether or not any given file is processed by Liquid depends on whether or not it has YAML front matter, essentially a (possibly empty) collection of metadata. On most content pages you’ll want to have at least a title attribute, while e.g. CSS files and images should stay as they are. Also index.md or index.html is processed according to this rule.
You can look at the full source code of this web site at GitHub.
One source file per post, e.g. 2016-06-10-internationalizing-autoit-applications-by-something-similar-to-gnu-gettext.html
The initial set was created by a conversion tool from a wordpress.com XML export. Later files, starting 2017, were hand edited. .md files in Markdown syntax are perfectly acceptable too.
These are directories to be copied verbatim to the output (_site directory).
css/styles.scss is automatically processed, i.e. converted to css/styles.css, by the SCSS stylesheet preprocessor that is built into Jekyll.
The webfonts are currently not used in the CSS, because Firefox in Windows renders them really ugly.
assets/ contains content-specific files such as screenshot images showing in blog posts.
images/ contains site-specific images such as the site (business) logo, a portrait foto etc.
]]>