Commit Graph

806 Commits

Author SHA1 Message Date
Cecylia Bocovich a85474e3e3 Update license 2020-04-13 16:40:54 -04:00
Cecylia Bocovich d33a70a47e Remove accidental files
These files are empty or were accidentially added to the repository
2020-04-13 16:38:51 -04:00
Cecylia Bocovich e96087d745 Remove GetTor website mirror code
This used to be the location of the code for gettor.torproject.org, but
that has been moved into a separate repository. While this old code is
suitable for GetTor mirror sites, it is very out of date and unclear
whether we want to support this going forward.
2020-04-13 16:31:03 -04:00
Cecylia Bocovich 880912bb2c Remove all old upload scripts
These have been replaced by newer scripts located in the top-level
scripts/ directory.
2020-04-13 16:28:24 -04:00
Cecylia Bocovich 63559201e2 Add signature verification instructions to email 2020-03-18 16:12:10 -04:00
Cecylia Bocovich 36820cc151 Update GetTor links message
This makes the following changes to the links message:
- Breaks user actions into steps
- Simplifies wordings
- Includes signature verification and bridge instructions
2020-03-18 14:57:45 -04:00
Cecylia Bocovich 7f1fd7e2c8 Update GetTor help message
This restructures the GetTor help message. The message now includes:
- instructions to reply to the message for links
- available locales
- a valid example links request
2020-03-18 14:36:23 -04:00
Cecylia Bocovich bd227cfc75 Update setup.py and requirements.txt 2020-03-18 13:33:25 -04:00
Cecylia Bocovich 829fa9fbaa Make locale parsing less eager
This will make the locale parsing more restrictive so that, for example,
sending "i like fries" will not result in the locale "fr".
2020-03-12 13:13:08 -04:00
Cecylia Bocovich c8e385babe Add more test cases for different email styles
teor mentioned in ticket #23225 that some clients will top-post replies
without quoting the the text. This commit adds a test case for that.
Since the command detector is greedy, gettor parses the reply first.

Also added a test for bottom-posting.
2020-03-12 13:13:08 -04:00
Cecylia Bocovich 0cd6443cdd Ensure replies to the help message will work
This commit makes some changes and adds some tests to ensure that
valid replies to the gettor help message will contain links.
Non-valid replies should still return the GetTor help message.
2020-03-12 13:13:00 -04:00
Cecylia Bocovich 0297dbcddb Change gitlab repository name to include version
This fixes an issue caused by gitlab repositories soft-delete for 7 days
before they are fully deleted. If we remove a repository in order
to upload new binaries, we need to be able to create a new one right
away. See https://bugs.torproject.org/33034
2020-03-11 12:46:50 -04:00
Cecylia Bocovich 61afa2c544 Always reply to gettor emails
This fixes a bug where improperly formatted emails were not receiving
gettor replies. We should always reply to emails and default to sending
a help message if the request was invalid.

This also prioritizes sending links if both a valid links request and
the word help were sent. Several tests were added.
2020-03-06 14:07:45 -05:00
Cecylia Bocovich 64df8b27b3 Expand tests for database functions in gettor 2020-02-19 14:40:28 -05:00
Cecylia Bocovich c413a8ebda Add method to close dbpool and call it
We were leaving connections to the database open, which causes some
calls to hang. This adds destructors to the SQLite3 class and the
classes that use it, and refactors some code to make one database
connection per class object. This also makes sure that the destructors
for objects that use databases are actually called.
2020-02-19 14:40:28 -05:00
Cecylia Bocovich c6cfd91241 Add database tests 2020-02-19 14:40:28 -05:00
Cecylia Bocovich dac0ab6263 Revert to using twisted for sqlite3 db 2020-02-19 14:36:56 -05:00
Cecylia Bocovich 3d6fb34e71 Add tests for database functions 2020-02-13 15:01:00 -05:00
Cecylia Bocovich e4f6d98fb9 Switch from twisted adbapi to sqlite3 package
The rest of GetTor uses the python sqlite3 package, and it's much
simpler to use and to test.
2020-02-07 19:15:38 -05:00
Cecylia Bocovich 40589f08d6 Removed unused imports 2020-02-07 19:15:38 -05:00
Cecylia Bocovich c2930fe339 Remove completed gettor requests from database
This will remove gettor requests from the requests database when they
are completed (rather than changing the status to 'SENT'). There's no
need to keep this information around, and we don't want to store
hashed email addresses as identifiers long term.
2020-02-07 19:12:31 -05:00
Cecylia Bocovich 3dfc2fdbaa Update help message to allow locales
The only help message urged users to only specify 'en' as a locale. We
now support all locales for tor browser binaries.
2020-01-31 11:52:23 -05:00
Cecylia Bocovich 56399f5a71 Update send mail to get user requested locale
This updates sendmail to get links and save stats for the user requested
locale. Right now all body translations default to english since we
haven't translated it yet.
2020-01-31 09:53:32 -05:00
Cecylia Bocovich 277ba71b46 Make locale parser more robust
This change expands the locale parse to have the following properties:
- if only the language code is given, choses the regionalization that
occurs first in the locale list (e.g., "en" --> "en-US"
- if regionalization for the language is *not* present, choses the
generalized language or a different regionalization (e.g. "pt-PT" -->
"pt-BR")
- parses both the subject and body looking for the most specific
regionalization
- defaults to en-US if no available language is found
2020-01-27 10:17:29 -05:00
Cecylia Bocovich 4fa4d6368c Refactor email keyword parser
This commit refactors build_request to prevent duplicate code. It
also moves the platforms definition to the parser constructor.
2020-01-27 09:58:26 -05:00
Cecylia Bocovich 187e27da76 Modified email parser to get locales from db
Before we were conflating the locales for the tor browser binaries with
the available locale for gettor email body localizations. This patch
checks our links database for available locales.
2020-01-27 09:48:23 -05:00
Cecylia Bocovich 4005315b36 Slight changes to README for tests 2020-01-23 13:32:56 -05:00
Cecylia Bocovich 9bbda2ab50 Change es string test to match json file
We haven't translated any es or pt strings, so use the English
translation for now until we update those files.
2020-01-23 13:32:56 -05:00
Cecylia Bocovich 3517e54ca7 Remove twitter test that requires auth
Removes tests that require access to secrets.
2020-01-23 13:32:56 -05:00
Cecylia Bocovich c1672a3f8d Create a test database with gettor links 2020-01-23 13:32:56 -05:00
Cecylia Bocovich 2ef6773bec Update tests to load a test config 2020-01-23 13:32:56 -05:00
Cecylia Bocovich 34e912bbb3 Fix edge case where config file never declared
This commit fixes an edge case in which the config filename is never
declared if the config is set but the file does not exist on disk.
2020-01-23 13:32:56 -05:00
Cecylia Bocovich 4c8dd8568a Make gitlab repositories public
Created projects default to private, we need to set the visibility in
the script to make them public.
2020-01-23 09:26:50 -05:00
Cecylia Bocovich 5404ef8183 Ensure failure is declared before referenced
This fixes a bug where, if there were no errors in uploading binaries,
failure is referenced before it was ever declared.
2020-01-23 09:14:15 -05:00
Cecylia Bocovich 1631494e1c Ensure base64 encoded binaries are in ascii
This fixes an issue that occurred while running the new gitlab script on
the gettor machine. Newer library versions require the b64 encoded byte
string to be converted to ascii text.
2020-01-23 09:05:55 -05:00
Cecylia Bocovich ff6379b556 Clean up script based on feedback
This commit cleans up the script a bit by:
- making more detailed comments,
- streamlining the choice of remote repository
- printing all output to std error
- captilization consistency
2020-01-22 14:58:05 -05:00
Cecylia Bocovich cbec273581 Removed git commands from update_files script
We now have a separate script for both GitLab and Github endpoints.
2020-01-22 14:58:05 -05:00
Cecylia Bocovich 8260a79d95 Modified scripts to add correct gitlab links to db 2020-01-22 14:58:05 -05:00
Cecylia Bocovich 78415a6d32 Renamed update_github to a more generic update_git 2020-01-22 14:58:05 -05:00
Cecylia Bocovich c63cb34499 Extend script to update gitlab remote
Updated and refactored the script to upload tor browser binaries to the
gitlab remote using the GitLab REST API.
2020-01-22 14:58:03 -05:00
Cecylia Bocovich 518c546738 Update links to new github repository 2020-01-17 16:44:29 -05:00
hiro 55b2cc2c86 Fix issues with duplicated filename in email body 2020-01-16 17:21:00 +01:00
hiro 906bec40b2 Fix current issue with gettor strings splitting #32906
Add tests to check sent email messages.
2020-01-10 23:04:06 +01:00
Cecylia Bocovich dfab6fc7ac Added a step to cleanup rclone remote
This fixes a bug where we exceed our google drive quota. Deleted files
were moved to a trash bin and not removed from storage entirely.
2020-01-10 11:02:23 -05:00
Cecylia Bocovich e6f6ad93b1 Updated github repo 2020-01-08 14:53:11 -05:00
Cecylia Bocovich 21138997b6 Update each tb binary individually
This modifies update_files to update each tor browser binary in gitlab,
internet archive, and google docs, and then delete the binary before
downloading the next one. This will save storage space on the gettor
host.
2020-01-07 13:30:22 -05:00
hiro ff07810650 Include feedback from review 2019-12-11 16:11:20 +01:00
hiro fb3155de15 Document scripts and service files 2019-12-11 16:06:44 +01:00
hiro b5a1cffb31 Remove sendmail cfg file - not used 2019-12-11 16:06:44 +01:00
hiro 744934523a Update readme
Breakdown strings for translations
Delete unused scripts
2019-12-11 16:06:44 +01:00