Sfoglia il codice sorgente

Merge https://github.com/Psiphon-Labs/psiphon-tunnel-core

Rod Hynes 5 anni fa
parent
commit
7e9ef82825

+ 5 - 2
ClientLibrary/README.md

@@ -4,9 +4,12 @@
 
 If you are planning to embed Psiphon in a mobile application, please use the [MobileLibrary](../MobileLibrary).
 
-## Usage
+## Using the Library in your App
 
-If you are using the Library in your app, please read the [USAGE.md](USAGE.md) instructions.
+**First step:** Review the sample code, located under `example`.
+This code provides an example of how to correctly use the client library.
+
+**Second step:** Review the comments for `Start` and `Stop` in [`PsiphonTunnel.go`](PsiphonTunnel.go). They describe the client interface.
 
 ## Building for Darwin (iOS, MacOS)
 

+ 0 - 16
ClientLibrary/USAGE.md

@@ -1,16 +0,0 @@
-# Using the Psiphon Client Library
-
-## Overview
-
-The Psiphon Client Library enables you to easily embed Psiphon in your app.
-
-## Using the Psiphon network
-
-In order to use the Psiphon Client Library over the Psiphon network, you need to contact Psiphon to obtain connection parameters to use with your application. Please email us at [info@psiphon.ca](mailto:info@psiphon.ca).
-
-## Using the Library in your App
-
-**First step:** Review the sample code, located under `example`.
-This code provides an example of how to correctly use the client library.
-
-**Second step:** Review the comments for `Start` and `Stop` in [`PsiphonTunnel.go`](PsiphonTunnel.go). They describe the client interface.

+ 0 - 8
ConsoleClient/README.md

@@ -42,11 +42,3 @@ bin
     └── psiphon-tunnel-core-x86_64.exe
 
 ```
-
-### Building without Docker
-
-See the [main README build section](../README.md#build)
-
-### Creating a configuration file
-
-See the [main README configuration section](../README.md#configure)

+ 30 - 43
MobileLibrary/Android/README.md

@@ -3,8 +3,36 @@
 ### Overview
 
 Psiphon Library for Android enables you to easily embed Psiphon in your Android
-app. The Psiphon Library for Android is implemented in Go and follows the standard
-conventions for using a Go library in an Android app.
+app.
+
+### Using the Library
+
+#### If building from source
+
+ 1. Build `ca.psiphon.aar` from via the docker container.
+ 2. Add `ca.psiphon.aar` to your Android Studio project as described in the [gomobile documentation](https://godoc.org/golang.org/x/mobile/cmd/gomobile)
+
+#### If using Maven based binary distribution
+
+1. Add maven repo to your app build.gradle
+```
+repositories {
+    ...
+    maven {
+        url "https://raw.github.com/Psiphon-Labs/psiphon-tunnel-core-Android-library/master"
+    }
+}
+```
+then add PsiphonTunnel dependency like following
+```
+dependencies {
+    ...
+    implementation 'ca.psiphon:psiphontunnel:x.y.z'
+}
+```
+Where x.y.z is the target version. Latest available release version can be found at https://github.com/Psiphon-Labs/psiphon-tunnel-core-Android-library
+
+See example usage in [TunneledWebView sample app](./SampleApps/TunneledWebView/README.md)
 
 ### Building with Docker
 
@@ -46,44 +74,3 @@ When that command completes, the compiled `.aar` files (suitable for use in an A
 
  1. Follow Go Android documentation ([gomobile documentation](https://godoc.org/golang.org/x/mobile/cmd/gomobile))
  2. Run `make.bash`
-
-### Using the Library
-
-#### If building from source
-
- 1. Build `ca.psiphon.aar` from via the docker container.
- 2. Add `ca.psiphon.aar` to your Android Studio project as described in the [gomobile documentation](https://godoc.org/golang.org/x/mobile/cmd/gomobile)
-
-#### If using Maven based binary distribution
-
-1. Add maven repo to your app build.gradle
-```
-repositories {
-    ...
-    maven {
-        url "https://raw.github.com/Psiphon-Labs/psiphon-tunnel-core-Android-library/master"
-    }
-}
-```
-then add PsiphonTunnel dependency like following
-```
-dependencies {
-    ...
-    implementation 'ca.psiphon:psiphontunnel:2.0.2'
-}
-```
-Where 2.0.2 is the target version. Latest available release version can be found at https://github.com/Psiphon-Labs/psiphon-tunnel-core-Android-library
-
-See example usage in [TunneledWebView sample app](./SampleApps/TunneledWebView/README.md)
-
----
-
-**NOTE**
-
-The maven repo at `https://raw.github.com/Psiphon-Labs/psiphon-tunnel-core-Android-library/master/releases` which contains releases up to v2.0.2 is deprecated and will not be updated. All new releases starting with v2.0.2 will be deployed to maven repo at  `https://raw.github.com/Psiphon-Labs/psiphon-tunnel-core-Android-library/master`
-
----
-
-##### Limitations
-
- - Only supports one concurrent instance of Psiphon.

+ 0 - 15
MobileLibrary/iOS/BUILD.md

@@ -1,7 +1,5 @@
 # Building the Psiphon iOS Library
 
-**Note:** If you want to use the Psiphon library, you must use the pre-built binary. See [USAGE.md](USAGE.md) for instructions. (This building doc is for Psiphon devs' reference.)
-
 ## Manual Build
 
 ### Prerequisites
@@ -25,16 +23,3 @@
 #### Testing
 
 Run `test-psiphon-framework.sh`.
-
-## Automatic Build -- Jenkins
-
-Build artifacts can be found in Jenkins.
-
-
-## Deployment
-
-* Version numbers are arbitrary, but should be [semver](http://semver.org/)-compatible.
-
-* iOS and Android Library builds should be done at the same time, from the code, with the same version number. (There may be exceptions to this, where only one platform release makes sense.)
-
-* Use Github Releases to publish the Library binaries. Create a tag on the correct commit hash with the name of the version. Create a Release with a zip file containing the `.framework` directory and the `USAGE.md` file. Attach the Android Library binary

+ 0 - 4
MobileLibrary/iOS/USAGE.md

@@ -9,10 +9,6 @@ blocked by censors.
 The Psiphon Library is available as a `.framework` that can be easily included
 in your project using these instructions.
 
-## Using the Psiphon network
-
-In order to use a Psiphon library over the Psiphon network, you need to contact Psiphon to obtain connection parameters to use with your application. Please email us at [info@psiphon.ca](mailto:info@psiphon.ca).
-
 ## Using the Library in your App
 
 **First step:** Review the sample app, located under `SampleApps`.

+ 69 - 58
README.md

@@ -1,78 +1,100 @@
 [![Build Status](https://travis-ci.org/Psiphon-Labs/psiphon-tunnel-core.png)](https://travis-ci.org/Psiphon-Labs/psiphon-tunnel-core) [![Coverage Status](https://coveralls.io/repos/github/Psiphon-Labs/psiphon-tunnel-core/badge.svg?branch=master)](https://coveralls.io/github/Psiphon-Labs/psiphon-tunnel-core?branch=master)
 
-Psiphon 3 Tunnel Core README
+Psiphon Tunnel Core README
 ================================================================================
 
 Overview
 --------------------------------------------------------------------------------
 
-Psiphon client and server components implemented in Go. These components provides core tunnel functionality, handling all aspects of evading blocking and relaying traffic through Psiphon. In the client, local proxies provide an interface for routing traffic through the tunnel.
+Psiphon is an Internet censorship circumvention system.
 
-The client component does not include a UI and does not handle capturing or routing local traffic. These major aspects are handled by other parts of Psiphon client applications.
+The tunnel core project includes tunneling clients and a server, which together implement all aspects of evading blocking and relaying traffic through Psiphon.
 
-Status
---------------------------------------------------------------------------------
+All Psiphon open source projects, including the complete open source code for Android, iOS, and Windows clients may be found at [www.github.com/Psiphon-Inc/psiphon](https://www.github.com/Psiphon-Inc/psiphon).
+
+For more information about Psiphon Inc., please visit our web site at [www.psiphon.ca](https://www.psiphon.ca).
 
-This project is in production and used as the tunneling engine in our Windows and Android clients, which are available at our [Psiphon 3 repository](https://bitbucket.org/psiphon/psiphon-circumvention-system).
+```
+psiphon-tunnel-core
+  └── ClientLibrary  General client libraries
+  └── ConsoleClient  CLI client program
+  └── MobileLibrary  Android/iOS client libraries
+  └── Server         Server program
+  └── psiphon        Client code package
+    └── common\...   Common code packages
+    └── server       Server code package
+```
 
-Client Setup
+
+Running Psiphon
 --------------------------------------------------------------------------------
 
-### Build
+### Get the programs
 
-* Go 1.13 (or higher) is required.
-* This project builds and runs on recent versions of Windows, Linux, and Mac OS X.
-* Note that the `psiphon` package is imported using the absolute path `github.com/Psiphon-Labs/psiphon-tunnel-core/psiphon`; without further local configuration, `go` will use this version of the code and not the local copy in the repository.
-* In this repository, run `go build` in `ConsoleClient` to make the `ConsoleClient` binary, a console Psiphon client application.
-  * Build versioning info may be configured as follows, and passed to `go build` in the `-ldflags` argument:
+Official binaries are avaiable at:
+- https://github.com/Psiphon-Labs/psiphon-tunnel-core-binaries
+- https://github.com/Psiphon-Labs/psiphon-tunnel-core/releases, for libraries
 
-    ```
-    BUILDDATE=$(date --iso-8601=seconds)
-    BUILDREPO=$(git config --get remote.origin.url)
-    BUILDREV=$(git rev-parse --short HEAD)
-    GOVERSION=$(go version | perl -ne '/go version (.*?) / && print $1')
-    DEPENDENCIES=$(echo -n "{" && go list -f '{{range $dep := .Deps}}{{printf "%s\n" $dep}}{{end}}' | xargs go list -f '{{if not .Standard}}{{.ImportPath}}{{end}}' | xargs -I pkg bash -c 'cd $GOPATH/src/pkg && echo -n "\"pkg\":\"$(git rev-parse --short HEAD)\","' | sed 's/,$/}/')
+For these instructions, use:
+- [psiphond](https://github.com/Psiphon-Labs/psiphon-tunnel-core-binaries/blob/master/psiphond/psiphond)
+- [ConsoleClient](https://github.com/Psiphon-Labs/psiphon-tunnel-core-binaries/blob/master/linux/psiphon-tunnel-core-x86_64)
 
-    LDFLAGS="\
-    -X github.com/Psiphon-Labs/psiphon-tunnel-core/psiphon/common/buildinfo.buildDate=$BUILDDATE \
-    -X github.com/Psiphon-Labs/psiphon-tunnel-core/psiphon/common/buildinfo.buildRepo=$BUILDREPO \
-    -X github.com/Psiphon-Labs/psiphon-tunnel-core/psiphon/common/buildinfo.buildRev=$BUILDREV \
-    -X github.com/Psiphon-Labs/psiphon-tunnel-core/psiphon/common/buildinfo.goVersion=$GOVERSION \
-    -X github.com/Psiphon-Labs/psiphon-tunnel-core/psiphon/common/buildinfo.dependencies=$DEPENDENCIES \
-    "
-    ```
+### Generate configuration data
 
-### Configure
+Run the "generate" mode of psiphond to generate configs, setting the IP address as appropriate; this is the address the client will use to connect to the server.
 
- * Configuration files are standard text files containing a valid JSON object. Example:
+```
+$ ./psiphond -ipaddress 127.0.0.1 -protocol OSSH:9999 -protocol generate
 
+$ ls
+psiphond
+psiphond.config
+psiphond-osl.config
+psiphond-tactics.config
+psiphond-traffic-rules.config
+server-entry.dat
+```
 
-  <!--BEGIN-SAMPLE-CONFIG-->
-  ```
-  {
-      "PropagationChannelId" : "<placeholder>",
-      "SponsorId" : "<placeholder>",
-      "LocalHttpProxyPort" : 8080,
-      "LocalSocksProxyPort" : 1080
-  }
-  ```
-  <!--END-SAMPLE-CONFIG-->
+Create a client config file, copying the contents of `server-entry.dat` to the `TargetServerEntry` field.
 
-*Note: The lines `<!--BEGIN-SAMPLE-CONFIG-->` and `<--END-SAMPLE-CONFIG-->` (visible in the raw Markdown) are used by the [config test](psiphon/config_test.go). Do not remove them.*
+```
+$ cat server-entry.dat 
+3132372e302e302e31202020207b22746167223a22222c2269[...]
 
-* All config file parameters are [documented here](https://godoc.org/github.com/Psiphon-Labs/psiphon-tunnel-core/psiphon#Config).
-* Replace each `<placeholder>` with a value from your Psiphon server. The Psiphon server-side stack is open source and can be found in our [Psiphon 3 repository](https://bitbucket.org/psiphon/psiphon-circumvention-system).
+$ cat client.config
+{
+    "LocalHttpProxyPort" : 8080,
+    "LocalSocksProxyPort" : 1080,
 
+    "PropagationChannelId" : "24BCA4EE20BEB92C",
+    "SponsorId" : "721AE60D76700F5A",
 
-### Run
+    "TargetServerEntry" : "3132372e302e302e31202020207b22746167223a22222c2269[...]"
+}
+```
 
-* Run `./ConsoleClient --config psiphon.config` where `psiphon.config` is created as described in the [Configure](#configure) section above
+### Run psiphond
 
+```
+$ ./psiphond run
+{"localAddress":"127.0.0.1:9999","msg":"listening","tunnelProtocol":"OSSH",[...]}
+{"localAddress":"127.0.0.1:9999","msg":"running","tunnelProtocol":"OSSH",[...]}
+[...]
+```
 
-Other Platforms
---------------------------------------------------------------------------------
+### Run the console client
 
-* The project builds and runs on Android. See the [Android Library README](MobileLibrary/Android/README.md) for more information about building the Go component, and the [Android Sample App README](MobileLibrary/Android/SampleApps/TunneledWebView/README.md) for a sample Android app that uses it.
+```
+$ ./ConsoleClient -config ./client.config
+{"data":{"port":1080},"noticeType":"ListeningSocksProxyPort",[...]}
+{"data":{"port":8080},"noticeType":"ListeningHttpProxyPort",[...]}
+[...]
+{"data":{"count":1},"noticeType":"Tunnels",[...]}
+```
+
+### Tunnel traffic through Psiphon
+
+Use the local SOCKS proxy (port 1080) or HTTP proxy (port 8080) to tunnel traffic.
 
 
 Acknowledgements
@@ -103,14 +125,3 @@ Psiphon Tunnel Core uses:
 * [cloudflare/tls-tris](https://github.com/cloudflare/tls-tris)
 * [Yawning/chacha20](https://github.com/Yawning/chacha20)
 * [wader/filtertransport](https://github.com/wader/filtertransport)
-
-Licensing
---------------------------------------------------------------------------------
-
-Please see the LICENSE file.
-
-
-Contacts
---------------------------------------------------------------------------------
-
-We maintain a developer mailing list at	<psiphon3-developers@googlegroups.com>. For more information about Psiphon Inc., please visit our web site at [www.psiphon.ca](http://www.psiphon.ca).

+ 1 - 7
Server/README.md

@@ -1,14 +1,8 @@
 ## Psiphon Tunnel Core Server README
 
-### Overview
-The `Server`/`psiphond` program and the `github.com/Psiphon-Labs/psiphon-tunnel-core/psiphon/server` package contain the Psiphon server software.
-
-Functionality is based on the [legacy server stack](https://bitbucket.org/psiphon/psiphon-circumvention-system/src/tip/Server/). `psiphond` has entered production.
-
 ### Build
 Prerequisites:
  - Go 1.13 or later
 
 Build Steps:
- - Build: `go build -o psiphond main.go` (will generate a binary named `psiphond` for Linux/OSX  or `psiphond.exe` for Windows)
-
+ - Build: `go build -o psiphond main.go`

+ 24 - 0
psiphon/common/osl/README.md

@@ -0,0 +1,24 @@
+## Psiphon Tunnel Obfuscated Server Lists
+
+### Design
+
+Obfuscated server lists (OSLs) is a mechanism to distribute Psiphon servers to select users. Like standard server lists, these server lists are publicly available at locations encoded in the Psiphon client. Unlike standard server lists, these lists are encrypted. Only clients that have the required keys can access the servers within these lists.
+
+One scheme for issuing server list encryption keys (SLOKs) is for the Psiphon servers to release the keys to clients that meet certain criteria for data transfer. The criteria is designed to distinguish regular, typical users from bots, scripts, and opportunistic adversaries.
+The data transfer SLOK scheme uses thresholds and groups of criteria to avoid being too strict. This serves two purposes: users don't need to meet all targets to earn a SLOK; and knowing that a user has earned a SLOK does not precisely reveal the user's activity.
+
+Parameters include:
+- The base time period for each SLOK. For example, with a base time period of one week, any sufficient network activity within a given week would earn the SLOKs. Possession of the SLOK does not reveal when, within the week the network, activity took place.
+- A set of network destinations for each SLOK, and a threshold. For example, sufficient data transfer to any 2 of 5 of a group of different network destinations, which could be defined to include typical "search", "social media", "messaging", "news", "e-commerce" destinations. Possession of the SLOK does not reveal which of the groups were visited; nor exactly which sites within the groups. Furthermore, the definition of network destinations admits all sites hosted on shared infrastructure, introducing further ambiguity.
+- A time threshold on top of SLOKs. For example, any 2 of 4 weekly SLOKs are required to decrypt a given OSL. Decrypting the OSL and using its servers does not reveal exactly which SLOKs were used.
+- A propagation channel ID for each SLOK. Each propagation channel has its own SLOK key space, and an adversary in possession of a SLOK from a different propagation channel cannot use that to determine the properties of SLOKs from other propagation channels.
+
+All parameters and criteria are operational secrets stored only in Psiphon servers/automation and not publicly revealed.
+
+Client progress towards a SLOK is stored in Psiphon server volatile memory (and, potentially, swap).
+
+When a SLOK is earned by a client, the Psiphon server sends the SLOK to the client through the secure Psiphon SSH tunnel. No logs are recorded for individual SLOK events.
+
+The client stores all its SLOKs in its local database. SLOKs are random values. An adversary that compromises a client may obtain the SLOKs, but does not have access to the parameters which generated the SLOK. And even if the parameters are compromised (or the adversary reverse engineers partial parameter information by earning its own SLOKs via random activity) the thresholds and groupings within the parameters mean a single SLOK does not demonstrate browsing a particular site or browsing at a particular time.
+
+When an OSL is downloaded, this event is logged. This is how we monitor the mechanism. This log is a standard Psiphon log, which does not record the client IP address but does record GeoIP information. A log that an OSL has been unlocked indirectly reveals that a client has sufficient SLOKs, earned through the various criteria. But this is much less specific than, for example, a domain bytes transferred log.

+ 4 - 4
psiphon/config.go

@@ -544,7 +544,7 @@ type Config struct {
 	// ApplicationParameters is for testing purposes.
 	ApplicationParameters parameters.KeyValues
 
-	// MigrateHompageNoticesFilename migrates a homepage file from the path
+	// MigrateHomepageNoticesFilename migrates a homepage file from the path
 	// previously configured with setNoticeFiles to the new path for homepage
 	// files under the data root directory. The file specified by this config
 	// value will be moved to config.GetHomePageFilename().
@@ -553,7 +553,7 @@ type Config struct {
 	// migrations are performed.
 	//
 	// If not set, no migration operation will be performed.
-	MigrateHompageNoticesFilename string
+	MigrateHomepageNoticesFilename string
 
 	// MigrateRotatingNoticesFilename migrates notice files from the path
 	// previously configured with setNoticeFiles to the new path for notice
@@ -1796,9 +1796,9 @@ func (n *loggingNetworkIDGetter) GetNetworkID() string {
 func migrationsFromLegacyNoticeFilePaths(config *Config) []common.FileMigration {
 	var noticeMigrations []common.FileMigration
 
-	if config.MigrateHompageNoticesFilename != "" {
+	if config.MigrateHomepageNoticesFilename != "" {
 		noticeMigrations = append(noticeMigrations, common.FileMigration{
-			OldPath: config.MigrateHompageNoticesFilename,
+			OldPath: config.MigrateHomepageNoticesFilename,
 			NewPath: config.GetHomePageFilename(),
 		})
 	}

+ 9 - 15
psiphon/config_test.go

@@ -25,7 +25,6 @@ import (
 	"io/ioutil"
 	"os"
 	"path/filepath"
-	"strings"
 	"testing"
 
 	"github.com/Psiphon-Labs/psiphon-tunnel-core/psiphon/common"
@@ -33,12 +32,6 @@ import (
 	"github.com/stretchr/testify/suite"
 )
 
-const (
-	_README              = "../README.md"
-	_README_CONFIG_BEGIN = "<!--BEGIN-SAMPLE-CONFIG-->"
-	_README_CONFIG_END   = "<!--END-SAMPLE-CONFIG-->"
-)
-
 type ConfigTestSuite struct {
 	suite.Suite
 	confStubBlob      []byte
@@ -48,13 +41,14 @@ type ConfigTestSuite struct {
 }
 
 func (suite *ConfigTestSuite) SetupSuite() {
-	readmeBlob, _ := ioutil.ReadFile(_README)
-	readmeString := string(readmeBlob)
-	readmeString = readmeString[strings.Index(readmeString, _README_CONFIG_BEGIN)+len(_README_CONFIG_BEGIN) : strings.Index(readmeString, _README_CONFIG_END)]
-	readmeString = strings.TrimSpace(readmeString)
-	readmeString = strings.Trim(readmeString, "`")
-
-	suite.confStubBlob = []byte(readmeString)
+	suite.confStubBlob = []byte(`
+	{
+	    "PropagationChannelId" : "<placeholder>",
+	    "SponsorId" : "<placeholder>",
+	    "LocalHttpProxyPort" : 8080,
+	    "LocalSocksProxyPort" : 1080
+	}
+	`)
 
 	var obj map[string]interface{}
 	json.Unmarshal(suite.confStubBlob, &obj)
@@ -390,7 +384,7 @@ func LoadConfigMigrateTest(oslDirChildrenPreMigration []FileTree, oslDirChildren
 	config := &Config{
 		DataRootDirectory:                            testDataRootDirectory,
 		MigrateRotatingNoticesFilename:               filepath.Join(testDirectory, oldRotatingNoticesFilename),
-		MigrateHompageNoticesFilename:                filepath.Join(testDirectory, oldHomepageNoticeFilename),
+		MigrateHomepageNoticesFilename:               filepath.Join(testDirectory, oldHomepageNoticeFilename),
 		MigrateDataStoreDirectory:                    oldDataStoreDirectory,
 		PropagationChannelId:                         "ABCDEFGH",
 		SponsorId:                                    "12345678",

+ 3 - 10
vendor/github.com/Psiphon-Labs/tls-tris/handshake_server.go

@@ -105,16 +105,9 @@ func (c *Conn) serverHandshake() error {
 			if !obfuscator.VerifyTLSPassthroughMessage(
 				c.config.PassthroughKey, hs.clientHello.random) {
 
-				// Legitimate, older clients that don't use passthrough messages will hit
-				// this case. Reduce false positive event logs with this heuristic: if
-				// isResume, the client sent a valid session ticket, so either the client
-				// sent a valid obfuscated session ticket proving knowledge of the
-				// obfuscation key, or the client previously connected and obtained a
-				// server-issued session ticket (this latter case shouldn't happen as the
-				// passthough message is now required for all connections; but isResume
-				// doesn't strictly mean the session ticket was _obfuscated_).
-				c.config.PassthroughLogInvalidMessage(
-					c.conn.RemoteAddr().String())
+				clientAddr := c.conn.RemoteAddr().String()
+				clientIP, _, _ := net.SplitHostPort(clientAddr)
+				c.config.PassthroughLogInvalidMessage(clientIP)
 
 				doPassthrough = true
 				err = errors.New("passthrough: invalid client random")

+ 3 - 3
vendor/vendor.json

@@ -141,10 +141,10 @@
 			"revisionTime": "2020-01-16T02:28:06Z"
 		},
 		{
-			"checksumSHA1": "CcrIyMnOlnDAZnt6c9KAzy/IndQ=",
+			"checksumSHA1": "vK+EtAIPYfEe9M9rSoGZYffUsxo=",
 			"path": "github.com/Psiphon-Labs/tls-tris",
-			"revision": "28577248f5cb7f01bf0570a8f5bd7777359f07da",
-			"revisionTime": "2020-05-01T12:56:31Z"
+			"revision": "16ab4b2ea797315638cb1ee75f438ef9502b617c",
+			"revisionTime": "2020-05-04T18:37:24Z"
 		},
 		{
 			"checksumSHA1": "30PBqj9BW03KCVqASvLg3bR+xYc=",