路由是什么呢?路由即是网络数据包在网络中的传输路径,或者说数据包在传输过程中所经过的网络节点,比如路由器,代理服务器之类的。
那像OkHttp3这样的网络库对于数据包的路由需要做些什么事呢?用户可以为终端设置代理服务器,HTTP/HTTPS代理或SOCK代理。OkHttp3中的路由相关逻辑,需要从系统中获取用户设置的代理服务器的地址,将HTTP请求转换为代理协议的数据包,发给代理服务器,然后等待代理服务器帮助完成了网络请求之后,从代理服务器读取响应数据返回给用户。只有这样,用户设置的代理才能生效。如果网络库无视用户设置的代理服务器,直接进行DNS并做网络请求,则用户设置的代理服务器不生效。
这里就来看一下OkHttp3中路由相关的处理。
路由选择
如同Internet上的其它设备一样,每个路由节点都有自己的IP地址,加上端口号,则可以确定唯一的路由服务。以域名描述的HTTP/HTTPS代理服务器地址,可能对应于多个实际的代理服务器主机,因而一个代理服务器可能包含有多条路由。而SOCK代理服务器,则有着唯一确定的IP地址和端口号。
OkHttp3借助于RouteSelector来选择路由节点,并维护路由的信息。
1public final class RouteSelector { 2 private final Address address; 3 private final RouteDatabase routeDatabase; 4 5 /* The most recently attempted route. */ 6 private Proxy lastProxy; 7 private InetSocketAddress lastInetSocketAddress; 8 9 /* State for negotiating the next proxy to use. */ 10 private List<Proxy> proxies = Collections.emptyList(); 11 private int nextProxyIndex; 12 13 /* State for negotiating the next socket address to use. */ 14 private List<InetSocketAddress> inetSocketAddresses = Collections.emptyList(); 15 private int nextInetSocketAddressIndex; 16 17 /* State for negotiating failed routes */ 18 private final List<Route> postponedRoutes = new ArrayList<>(); 19 20 public RouteSelector(Address address, RouteDatabase routeDatabase) { 21 this.address = address; 22 this.routeDatabase = routeDatabase; 23 24 resetNextProxy(address.url(), address.proxy()); 25 } 26 27 /** 28 * Returns true if there's another route to attempt. Every address has at least one route. 29 */ 30 public boolean hasNext() { 31 return hasNextInetSocketAddress() 32 || hasNextProxy() 33 || hasNextPostponed(); 34 } 35 36 public Route next() throws IOException { 37 // Compute the next route to attempt. 38 if (!hasNextInetSocketAddress()) { 39 if (!hasNextProxy()) { 40 if (!hasNextPostponed()) { 41 throw new NoSuchElementException(); 42 } 43 return nextPostponed(); 44 } 45 lastProxy = nextProxy(); 46 } 47 lastInetSocketAddress = nextInetSocketAddress(); 48 49 Route route = new Route(address, lastProxy, lastInetSocketAddress); 50 if (routeDatabase.shouldPostpone(route)) { 51 postponedRoutes.add(route); 52 // We will only recurse in order to skip previously failed routes. They will be tried last. 53 return next(); 54 } 55 56 return route; 57 } 58 59 /** 60 * Clients should invoke this method when they encounter a connectivity failure on a connection 61 * returned by this route selector. 62 */ 63 public void connectFailed(Route failedRoute, IOException failure) { 64 if (failedRoute.proxy().type() != Proxy.Type.DIRECT && address.proxySelector() != null) { 65 // Tell the proxy selector when we fail to connect on a fresh connection. 66 address.proxySelector().connectFailed( 67 address.url().uri(), failedRoute.proxy().address(), failure); 68 } 69 70 routeDatabase.failed(failedRoute); 71 } 72 73 /** Prepares the proxy servers to try. */ 74 private void resetNextProxy(HttpUrl url, Proxy proxy) { 75 if (proxy != null) { 76 // If the user specifies a proxy, try that and only that. 77 proxies = Collections.singletonList(proxy); 78 } else { 79 // Try each of the ProxySelector choices until one connection succeeds. If none succeed 80 // then we'll try a direct connection below. 81 proxies = new ArrayList<>(); 82 List<Proxy> selectedProxies = address.proxySelector().select(url.uri()); 83 if (selectedProxies != null) proxies.addAll(selectedProxies); 84 // Finally try a direct connection. We only try it once! 85 proxies.removeAll(Collections.singleton(Proxy.NO_PROXY)); 86 proxies.add(Proxy.NO_PROXY); 87 } 88 nextProxyIndex = 0; 89 } 90 91 /** Returns true if there's another proxy to try. */ 92 private boolean hasNextProxy() { 93 return nextProxyIndex < proxies.size(); 94 } 95 96 /** Returns the next proxy to try. May be PROXY.NO_PROXY but never null. */ 97 private Proxy nextProxy() throws IOException { 98 if (!hasNextProxy()) { 99 throw new SocketException("No route to " + address.url().host() 100 + "; exhausted proxy configurations: " + proxies); 101 } 102 Proxy result = proxies.get(nextProxyIndex++); 103 resetNextInetSocketAddress(result); 104 return result; 105 } 106 107 /** Prepares the socket addresses to attempt for the current proxy or host. */ 108 private void resetNextInetSocketAddress(Proxy proxy) throws IOException { 109 // Clear the addresses. Necessary if getAllByName() below throws! 110 inetSocketAddresses = new ArrayList<>(); 111 112 String socketHost; 113 int socketPort; 114 if (proxy.type() == Proxy.Type.DIRECT || proxy.type() == Proxy.Type.SOCKS) { 115 socketHost = address.url().host(); 116 socketPort = address.url().port(); 117 } else { 118 SocketAddress proxyAddress = proxy.address(); 119 if (!(proxyAddress instanceof InetSocketAddress)) { 120 throw new IllegalArgumentException( 121 "Proxy.address() is not an " + "InetSocketAddress: " + proxyAddress.getClass()); 122 } 123 InetSocketAddress proxySocketAddress = (InetSocketAddress) proxyAddress; 124 socketHost = getHostString(proxySocketAddress); 125 socketPort = proxySocketAddress.getPort(); 126 } 127 128 if (socketPort < 1 || socketPort > 65535) { 129 throw new SocketException("No route to " + socketHost + ":" + socketPort 130 + "; port is out of range"); 131 } 132 133 if (proxy.type() == Proxy.Type.SOCKS) { 134 inetSocketAddresses.add(InetSocketAddress.createUnresolved(socketHost, socketPort)); 135 } else { 136 // Try each address for best behavior in mixed IPv4/IPv6 environments. 137 List<InetAddress> addresses = address.dns().lookup(socketHost); 138 for (int i = 0, size = addresses.size(); i < size; i++) { 139 InetAddress inetAddress = addresses.get(i); 140 inetSocketAddresses.add(new InetSocketAddress(inetAddress, socketPort)); 141 } 142 } 143 144 nextInetSocketAddressIndex = 0; 145 } 146 147 /** 148 * Obtain a "host" from an {@link InetSocketAddress}. This returns a string containing either an 149 * actual host name or a numeric IP address. 150 */ 151 // Visible for testing 152 static String getHostString(InetSocketAddress socketAddress) { 153 InetAddress address = socketAddress.getAddress(); 154 if (address == null) { 155 // The InetSocketAddress was specified with a string (either a numeric IP or a host name). If 156 // it is a name, all IPs for that name should be tried. If it is an IP address, only that IP 157 // address should be tried. 158 return socketAddress.getHostName(); 159 } 160 // The InetSocketAddress has a specific address: we should only try that address. Therefore we 161 // return the address and ignore any host name that may be available. 162 return address.getHostAddress(); 163 } 164 165 /** Returns true if there's another socket address to try. */ 166 private boolean hasNextInetSocketAddress() { 167 return nextInetSocketAddressIndex < inetSocketAddresses.size(); 168 } 169 170 /** Returns the next socket address to try. */ 171 private InetSocketAddress nextInetSocketAddress() throws IOException { 172 if (!hasNextInetSocketAddress()) { 173 throw new SocketException("No route to " + address.url().host() 174 + "; exhausted inet socket addresses: " + inetSocketAddresses); 175 } 176 return inetSocketAddresses.get(nextInetSocketAddressIndex++); 177 } 178 179 /** Returns true if there is another postponed route to try. */ 180 private boolean hasNextPostponed() { 181 return !postponedRoutes.isEmpty(); 182 } 183 184 /** Returns the next postponed route to try. */ 185 private Route nextPostponed() { 186 return postponedRoutes.remove(0); 187 } 188}
RouteSelector主要做了这样一些事情:
- 在
RouteSelector对象创建时,获取并保存用户设置的所有的代理。这里主要通过ProxySelector,根据uri来得到系统中的所有代理,并保存在Proxy列表proxies中。 - 给调用者提供接口,来选择可用的路由。调用者通过next()可以获取
RouteSelector中维护的下一个可用路由。调用者在连接失败时,可以再次调用这个接口来获取下一个路由。这个接口会逐个地返回每个代理的每个代理主机服务给调用者。在所有的代理的每个代理主机都被访问过了之后,还会返回曾经连接失败的路由。 - 维护路由节点的信息。
RouteDatabase用于维护连接失败的路由的信息,以避免浪费时间去连接一些不可用的路由。RouteDatabase中的路由信息主要由RouteSelector来维护。
RouteDatabase是一个简单的容器:
1package okhttp3.internal.connection; 2 3import java.util.LinkedHashSet; 4import java.util.Set; 5import okhttp3.Route; 6 7/** 8 * A blacklist of failed routes to avoid when creating a new connection to a target address. This is 9 * used so that OkHttp can learn from its mistakes: if there was a failure attempting to connect to 10 * a specific IP address or proxy server, that failure is remembered and alternate routes are 11 * preferred. 12 */ 13public final class RouteDatabase { 14 private final Set<Route> failedRoutes = new LinkedHashSet<>(); 15 16 /** Records a failure connecting to {@code failedRoute}. */ 17 public synchronized void failed(Route failedRoute) { 18 failedRoutes.add(failedRoute); 19 } 20 21 /** Records success connecting to {@code failedRoute}. */ 22 public synchronized void connected(Route route) { 23 failedRoutes.remove(route); 24 } 25 26 /** Returns true if {@code route} has failed recently and should be avoided. */ 27 public synchronized boolean shouldPostpone(Route route) { 28 return failedRoutes.contains(route); 29 } 30}
OkHttp3主要用(Address, Proxy, InetSocketAddress)的三元组来描述路由信息:
1package okhttp3; 2 3import java.net.InetSocketAddress; 4import java.net.Proxy; 5 6/** 7 * The concrete route used by a connection to reach an abstract origin server. When creating a 8 * connection the client has many options: 9 * 10 * <ul> 11 * <li><strong>HTTP proxy:</strong> a proxy server may be explicitly configured for the client. 12 * Otherwise the {@linkplain java.net.ProxySelector proxy selector} is used. It may return 13 * multiple proxies to attempt. 14 * <li><strong>IP address:</strong> whether connecting directly to an origin server or a proxy, 15 * opening a socket requires an IP address. The DNS server may return multiple IP addresses 16 * to attempt. 17 * </ul> 18 * 19 * <p>Each route is a specific selection of these options. 20 */ 21public final class Route { 22 final Address address; 23 final Proxy proxy; 24 final InetSocketAddress inetSocketAddress; 25 26 public Route(Address address, Proxy proxy, InetSocketAddress inetSocketAddress) { 27 if (address == null) { 28 throw new NullPointerException("address == null"); 29 } 30 if (proxy == null) { 31 throw new NullPointerException("proxy == null"); 32 } 33 if (inetSocketAddress == null) { 34 throw new NullPointerException("inetSocketAddress == null"); 35 } 36 this.address = address; 37 this.proxy = proxy; 38 this.inetSocketAddress = inetSocketAddress; 39 } 40 41 public Address address() { 42 return address; 43 } 44 45 /** 46 * Returns the {@link Proxy} of this route. 47 * 48 * <strong>Warning:</strong> This may disagree with {@link Address#proxy} when it is null. When 49 * the address's proxy is null, the proxy selector is used. 50 */ 51 public Proxy proxy() { 52 return proxy; 53 } 54 55 public InetSocketAddress socketAddress() { 56 return inetSocketAddress; 57 } 58 59 /** 60 * Returns true if this route tunnels HTTPS through an HTTP proxy. See <a 61 * href="http://www.ietf.org/rfc/rfc2817.txt">RFC 2817, Section 5.2</a>. 62 */ 63 public boolean requiresTunnel() { 64 return address.sslSocketFactory != null && proxy.type() == Proxy.Type.HTTP; 65 } 66 67 @Override public boolean equals(Object obj) { 68 if (obj instanceof Route) { 69 Route other = (Route) obj; 70 return address.equals(other.address) 71 && proxy.equals(other.proxy) 72 && inetSocketAddress.equals(other.inetSocketAddress); 73 } 74 return false; 75 } 76 77 @Override public int hashCode() { 78 int result = 17; 79 result = 31 * result + address.hashCode(); 80 result = 31 * result + proxy.hashCode(); 81 result = 31 * result + inetSocketAddress.hashCode(); 82 return result; 83 } 84}
在StreamAllocation中建立连接时,会通过RouteSelector获取可用路由。
在OkHttp3中,ProxySelector对象主要由OkHttpClient维护。
1public class OkHttpClient implements Cloneable, Call.Factory { 2...... 3 final ProxySelector proxySelector; 4 5 private OkHttpClient(Builder builder) { 6 this.dispatcher = builder.dispatcher; 7 this.proxy = builder.proxy; 8 this.protocols = builder.protocols; 9 this.connectionSpecs = builder.connectionSpecs; 10 this.interceptors = Util.immutableList(builder.interceptors); 11 this.networkInterceptors = Util.immutableList(builder.networkInterceptors); 12 this.proxySelector = builder.proxySelector; 13 14...... 15 16 public ProxySelector proxySelector() { 17 return proxySelector; 18 } 19 20...... 21 22 public Builder() { 23 dispatcher = new Dispatcher(); 24 protocols = DEFAULT_PROTOCOLS; 25 connectionSpecs = DEFAULT_CONNECTION_SPECS; 26 proxySelector = ProxySelector.getDefault(); 27 28...... 29 30 Builder(OkHttpClient okHttpClient) { 31 this.dispatcher = okHttpClient.dispatcher; 32 this.proxy = okHttpClient.proxy; 33 this.protocols = okHttpClient.protocols; 34 this.connectionSpecs = okHttpClient.connectionSpecs; 35 this.interceptors.addAll(okHttpClient.interceptors); 36 this.networkInterceptors.addAll(okHttpClient.networkInterceptors); 37 this.proxySelector = okHttpClient.proxySelector;
在创建OkHttpClient时,可以通过为OkHttpClient.Builder设置ProxySelector来定制ProxySelector。若没有指定,则所有的为默认ProxySelector。OpenJDK 1.8版默认的ProxySelector为sun.net.spi.DefaultProxySelector:
1public abstract class ProxySelector { 2 /** 3 * The system wide proxy selector that selects the proxy server to 4 * use, if any, when connecting to a remote object referenced by 5 * an URL. 6 * 7 * @see #setDefault(ProxySelector) 8 */ 9 private static ProxySelector theProxySelector; 10 11 static { 12 try { 13 Class<?> c = Class.forName("sun.net.spi.DefaultProxySelector"); 14 if (c != null && ProxySelector.class.isAssignableFrom(c)) { 15 theProxySelector = (ProxySelector) c.newInstance(); 16 } 17 } catch (Exception e) { 18 theProxySelector = null; 19 } 20 } 21 22 /** 23 * Gets the system-wide proxy selector. 24 * 25 * @throws SecurityException 26 * If a security manager has been installed and it denies 27 * {@link NetPermission}{@code ("getProxySelector")} 28 * @see #setDefault(ProxySelector) 29 * @return the system-wide {@code ProxySelector} 30 * @since 1.5 31 */ 32 public static ProxySelector getDefault() { 33 SecurityManager sm = System.getSecurityManager(); 34 if (sm != null) { 35 sm.checkPermission(SecurityConstants.GET_PROXYSELECTOR_PERMISSION); 36 } 37 return theProxySelector; 38 }
在Android平台上,默认ProxySelector所用的则是另外的实现:
1public abstract class ProxySelector { 2 3 private static ProxySelector defaultSelector = new ProxySelectorImpl(); 4 5 /** 6 * Returns the default proxy selector, or null if none exists. 7 */ 8 public static ProxySelector getDefault() { 9 return defaultSelector; 10 } 11 12 /** 13 * Sets the default proxy selector. If {@code selector} is null, the current 14 * proxy selector will be removed. 15 */ 16 public static void setDefault(ProxySelector selector) { 17 defaultSelector = selector; 18 }
Android平台下,默认的ProxySelector ProxySelectorImpl,其实现(不同版本的Android,实现不同,这里是android-6.0.1_r61的实现)如下:
1package java.net; 2import java.io.IOException; 3import java.util.Collections; 4import java.util.List; 5final class ProxySelectorImpl extends ProxySelector { 6 @Override public void connectFailed(URI uri, SocketAddress sa, IOException ioe) { 7 if (uri == null || sa == null || ioe == null) { 8 throw new IllegalArgumentException(); 9 } 10 } 11 @Override public List<Proxy> select(URI uri) { 12 return Collections.singletonList(selectOneProxy(uri)); 13 } 14 private Proxy selectOneProxy(URI uri) { 15 if (uri == null) { 16 throw new IllegalArgumentException("uri == null"); 17 } 18 String scheme = uri.getScheme(); 19 if (scheme == null) { 20 throw new IllegalArgumentException("scheme == null"); 21 } 22 int port = -1; 23 Proxy proxy = null; 24 String nonProxyHostsKey = null; 25 boolean httpProxyOkay = true; 26 if ("http".equalsIgnoreCase(scheme)) { 27 port = 80; 28 nonProxyHostsKey = "http.nonProxyHosts"; 29 proxy = lookupProxy("http.proxyHost", "http.proxyPort", Proxy.Type.HTTP, port); 30 } else if ("https".equalsIgnoreCase(scheme)) { 31 port = 443; 32 nonProxyHostsKey = "https.nonProxyHosts"; // RI doesn't support this 33 proxy = lookupProxy("https.proxyHost", "https.proxyPort", Proxy.Type.HTTP, port); 34 } else if ("ftp".equalsIgnoreCase(scheme)) { 35 port = 80; // not 21 as you might guess 36 nonProxyHostsKey = "ftp.nonProxyHosts"; 37 proxy = lookupProxy("ftp.proxyHost", "ftp.proxyPort", Proxy.Type.HTTP, port); 38 } else if ("socket".equalsIgnoreCase(scheme)) { 39 httpProxyOkay = false; 40 } else { 41 return Proxy.NO_PROXY; 42 } 43 if (nonProxyHostsKey != null 44 && isNonProxyHost(uri.getHost(), System.getProperty(nonProxyHostsKey))) { 45 return Proxy.NO_PROXY; 46 } 47 if (proxy != null) { 48 return proxy; 49 } 50 if (httpProxyOkay) { 51 proxy = lookupProxy("proxyHost", "proxyPort", Proxy.Type.HTTP, port); 52 if (proxy != null) { 53 return proxy; 54 } 55 } 56 proxy = lookupProxy("socksProxyHost", "socksProxyPort", Proxy.Type.SOCKS, 1080); 57 if (proxy != null) { 58 return proxy; 59 } 60 return Proxy.NO_PROXY; 61 } 62 /** 63 * Returns the proxy identified by the {@code hostKey} system property, or 64 * null. 65 */ 66 private Proxy lookupProxy(String hostKey, String portKey, Proxy.Type type, int defaultPort) { 67 String host = System.getProperty(hostKey); 68 if (host == null || host.isEmpty()) { 69 return null; 70 } 71 int port = getSystemPropertyInt(portKey, defaultPort); 72 return new Proxy(type, InetSocketAddress.createUnresolved(host, port)); 73 } 74 private int getSystemPropertyInt(String key, int defaultValue) { 75 String string = System.getProperty(key); 76 if (string != null) { 77 try { 78 return Integer.parseInt(string); 79 } catch (NumberFormatException ignored) { 80 } 81 } 82 return defaultValue; 83 } 84 /** 85 * Returns true if the {@code nonProxyHosts} system property pattern exists 86 * and matches {@code host}. 87 */ 88 private boolean isNonProxyHost(String host, String nonProxyHosts) { 89 if (host == null || nonProxyHosts == null) { 90 return false; 91 } 92 // construct pattern 93 StringBuilder patternBuilder = new StringBuilder(); 94 for (int i = 0; i < nonProxyHosts.length(); i++) { 95 char c = nonProxyHosts.charAt(i); 96 switch (c) { 97 case '.': 98 patternBuilder.append("\\."); 99 break; 100 case '*': 101 patternBuilder.append(".*"); 102 break; 103 default: 104 patternBuilder.append(c); 105 } 106 } 107 // check whether the host is the nonProxyHosts. 108 String pattern = patternBuilder.toString(); 109 return host.matches(pattern); 110 } 111}
可以看到,在Android平台上,主要是从System properties中获取的代理服务器的主机及其端口号,会过滤掉不能进行代理的主机的访问。
回到OkHttp中,在RetryAndFollowUpInterceptor中,创建Address对象时,从OkHttpClient对象获取ProxySelector。Address对象会被用于创建StreamAllocation对象,StreamAllocation在建立连接时,从Address对象中获取ProxySelector以选择路由。
1public final class RetryAndFollowUpInterceptor implements Interceptor { 2...... 3 private Address createAddress(HttpUrl url) { 4 SSLSocketFactory sslSocketFactory = null; 5 HostnameVerifier hostnameVerifier = null; 6 CertificatePinner certificatePinner = null; 7 if (url.isHttps()) { 8 sslSocketFactory = client.sslSocketFactory(); 9 hostnameVerifier = client.hostnameVerifier(); 10 certificatePinner = client.certificatePinner(); 11 } 12 13 return new Address(url.host(), url.port(), client.dns(), client.socketFactory(), 14 sslSocketFactory, hostnameVerifier, certificatePinner, client.proxyAuthenticator(), 15 client.proxy(), client.protocols(), client.connectionSpecs(), client.proxySelector()); 16 }
代理协议
OkHttp3发送给HTTP代理服务器的HTTP请求,与直接发送给HTTP服务器的HTTP请求有什么样的区别呢,还是说两者其实毫无差别呢?也就是HTTP代理的协议是什么样的呢?这里我们就通过对代码进行分析来仔细地看一下。
如我们在OkHttp3 HTTP请求执行流程分析中看到的,OkHttp3对HTTP请求是通过Interceptor链来处理的。 RetryAndFollowUpInterceptor创建StreamAllocation对象,处理http的重定向及出错重试。对后续Interceptor的执行的影响为修改Request并创建StreamAllocation对象。 BridgeInterceptor补全缺失的一些http header。对后续Interceptor的执行的影响主要为修改了Request。 CacheInterceptor处理http缓存。对后续Interceptor的执行的影响为,若缓存中有所需请求的响应,则后续Interceptor不再执行。 ConnectInterceptor借助于前面分配的StreamAllocation对象建立与服务器之间的连接,并选定交互所用的协议是HTTP 1.1还是HTTP 2。对后续Interceptor的执行的影响为,创建了HttpStream和connection。 CallServerInterceptor作为Interceptor链中的最后一个Interceptor,用于处理IO,与服务器进行数据交换。
OkHttp3对代理的处理是在ConnectInterceptor和CallServerInterceptor中完成的。再来看ConnectInterceptor的定义:
1package okhttp3.internal.connection; 2 3import java.io.IOException; 4import okhttp3.Interceptor; 5import okhttp3.OkHttpClient; 6import okhttp3.Request; 7import okhttp3.Response; 8import okhttp3.internal.http.HttpCodec; 9import okhttp3.internal.http.RealInterceptorChain; 10 11/** Opens a connection to the target server and proceeds to the next interceptor. */ 12public final class ConnectInterceptor implements Interceptor { 13 public final OkHttpClient client; 14 15 public ConnectInterceptor(OkHttpClient client) { 16 this.client = client; 17 } 18 19 @Override public Response intercept(Chain chain) throws IOException { 20 RealInterceptorChain realChain = (RealInterceptorChain) chain; 21 Request request = realChain.request(); 22 StreamAllocation streamAllocation = realChain.streamAllocation(); 23 24 // We need the network to satisfy this request. Possibly for validating a conditional GET. 25 boolean doExtensiveHealthChecks = !request.method().equals("GET"); 26 HttpCodec httpCodec = streamAllocation.newStream(client, doExtensiveHealthChecks); 27 RealConnection connection = streamAllocation.connection(); 28 29 return realChain.proceed(request, streamAllocation, httpCodec, connection); 30 } 31}
ConnectInterceptor利用前面的Interceptor创建的StreamAllocation对象,创建stream HttpCodec,以及RealConnection connection。然后把这些对象传给链中后继的Interceptor,也就是CallServerInterceptor处理。
为了厘清StreamAllocation的两个操作的详细执行过程,这里再回过头来看一下StreamAllocation对象的创建。StreamAllocation对象在RetryAndFollowUpInterceptor中创建:
1 @Override public Response intercept(Chain chain) throws IOException { 2 Request request = chain.request(); 3 4 streamAllocation = new StreamAllocation( 5 client.connectionPool(), createAddress(request.url()), callStackTrace);
创建StreamAllocation对象时,传入的ConnectionPool来自于OkHttpClient,创建的Address主要用于描述HTTP服务的目标地址相关的信息。
1public final class StreamAllocation { 2 public final Address address; 3 private Route route; 4 private final ConnectionPool connectionPool; 5 private final Object callStackTrace; 6 7 // State guarded by connectionPool. 8 private final RouteSelector routeSelector; 9 private int refusedStreamCount; 10 private RealConnection connection; 11 private boolean released; 12 private boolean canceled; 13 private HttpCodec codec; 14 15 public StreamAllocation(ConnectionPool connectionPool, Address address, Object callStackTrace) { 16 this.connectionPool = connectionPool; 17 this.address = address; 18 this.routeSelector = new RouteSelector(address, routeDatabase()); 19 this.callStackTrace = callStackTrace; 20 }
创建StreamAllocation对象时,除了创建RouteSelector之外,并没有其它特别的地方。
然后来看ConnectInterceptor中用来创建HttpCodec的newStream()方法:
1public final class StreamAllocation { 2 3...... 4 5 public HttpCodec newStream(OkHttpClient client, boolean doExtensiveHealthChecks) { 6 int connectTimeout = client.connectTimeoutMillis(); 7 int readTimeout = client.readTimeoutMillis(); 8 int writeTimeout = client.writeTimeoutMillis(); 9 boolean connectionRetryEnabled = client.retryOnConnectionFailure(); 10 11 try { 12 RealConnection resultConnection = findHealthyConnection(connectTimeout, readTimeout, 13 writeTimeout, connectionRetryEnabled, doExtensiveHealthChecks); 14 15 HttpCodec resultCodec; 16 if (resultConnection.http2Connection != null) { 17 resultCodec = new Http2Codec(client, this, resultConnection.http2Connection); 18 } else { 19 resultConnection.socket().setSoTimeout(readTimeout); 20 resultConnection.source.timeout().timeout(readTimeout, MILLISECONDS); 21 resultConnection.sink.timeout().timeout(writeTimeout, MILLISECONDS); 22 resultCodec = new Http1Codec( 23 client, this, resultConnection.source, resultConnection.sink); 24 } 25 26 synchronized (connectionPool) { 27 codec = resultCodec; 28 return resultCodec; 29 } 30 } catch (IOException e) { 31 throw new RouteException(e); 32 } 33 }
这个方法的执行流程为:
- 建立连接。 通过调用findHealthyConnection()方法来建立连接,后面我们通过分析这个方法的实现来了解连接的具体含义。
- 用前面创建的连接来创建HttpCodec。 对于HTTP/1.1创建Http1Codec,对于HTTP/2则创建Http2Codec。HttpCodec用于处理与HTTP具体协议相关的部分。比如HTTP/1.1是基于文本的协议,而HTTP/2则是基于二进制格式的协议,HttpCodec用于将请求编码为对应协议要求的传输格式,并在得到响应时,对数据进行解码。
然后来看findHealthyConnection()中创建连接的过程:
1 /** 2 * Finds a connection and returns it if it is healthy. If it is unhealthy the process is repeated 3 * until a healthy connection is found. 4 */ 5 private RealConnection findHealthyConnection(int connectTimeout, int readTimeout, 6 int writeTimeout, boolean connectionRetryEnabled, boolean doExtensiveHealthChecks) 7 throws IOException { 8 while (true) { 9 RealConnection candidate = findConnection(connectTimeout, readTimeout, writeTimeout, 10 connectionRetryEnabled); 11 12 // If this is a brand new connection, we can skip the extensive health checks. 13 synchronized (connectionPool) { 14 if (candidate.successCount == 0) { 15 return candidate; 16 } 17 } 18 19 // Do a (potentially slow) check to confirm that the pooled connection is still good. If it 20 // isn't, take it out of the pool and start again. 21 if (!candidate.isHealthy(doExtensiveHealthChecks)) { 22 noNewStreams(); 23 continue; 24 } 25 26 return candidate; 27 } 28 }
在这个方法中,是找到一个连接,然后判断其是否可用。如果可用则将找到的连接返回给调用者,否则寻找下一个连接。寻找连接可能是建立一个新的连接,也可能是复用连接池中的一个连接。
接着来看寻找连接的过程findConnection():
1 /** 2 * Returns a connection to host a new stream. This prefers the existing connection if it exists, 3 * then the pool, finally building a new connection. 4 */ 5 private RealConnection findConnection(int connectTimeout, int readTimeout, int writeTimeout, 6 boolean connectionRetryEnabled) throws IOException { 7 Route selectedRoute; 8 synchronized (connectionPool) { 9 if (released) throw new IllegalStateException("released"); 10 if (codec != null) throw new IllegalStateException("codec != null"); 11 if (canceled) throw new IOException("Canceled"); 12 13 RealConnection allocatedConnection = this.connection; 14 if (allocatedConnection != null && !allocatedConnection.noNewStreams) { 15 return allocatedConnection; 16 } 17 18 // Attempt to get a connection from the pool. 19 RealConnection pooledConnection = Internal.instance.get(connectionPool, address, this); 20 if (pooledConnection != null) { 21 this.connection = pooledConnection; 22 return pooledConnection; 23 } 24 25 selectedRoute = route; 26 } 27 28 if (selectedRoute == null) { 29 selectedRoute = routeSelector.next(); 30 synchronized (connectionPool) { 31 route = selectedRoute; 32 refusedStreamCount = 0; 33 } 34 } 35 RealConnection newConnection = new RealConnection(selectedRoute); 36 37 synchronized (connectionPool) { 38 acquire(newConnection); 39 Internal.instance.put(connectionPool, newConnection); 40 this.connection = newConnection; 41 if (canceled) throw new IOException("Canceled"); 42 } 43 44 newConnection.connect(connectTimeout, readTimeout, writeTimeout, address.connectionSpecs(), 45 connectionRetryEnabled); 46 routeDatabase().connected(newConnection.route()); 47 48 return newConnection; 49 }
这个过程大体为:
- 检查上次分配的连接是否可用,若可用则,则将上次分配的连接返回给调用者。
- 上次分配的连接不存在,或不可用,则从连接池中查找一个连接,查找的依据就是Address,也就是连接的对端地址,以及路由等信息。Internal.instance指向OkHttpClient的一个内部类的对象,Internal.instance.get()实际会通过ConnectionPool的
get(Address address, StreamAllocation streamAllocation)方法来尝试获取RealConnection。 若能从连接池中找到所需要的连接,则将连接返回给调用者。 - 从连接池中没有找到所需要的连接,则会首先选择路由。
- 然后创建新的连接RealConnection对象。
- acquire新创建的连接RealConnection对象,并将它放进连接池。不太确定这个地方的synchronized是不是太长了。貌似只有Internal.instance.put(connectionPool, newConnection)涉及到了全局对象的访问,而其它操作并没有。
- 调用newConnection.connect()建立连接。
这里再来看一下在ConnectionPool的get()操作执行的过程:
1 private final Deque<RealConnection> connections = new ArrayDeque<>(); 2 final RouteDatabase routeDatabase = new RouteDatabase(); 3 boolean cleanupRunning; 4 5 /** Returns a recycled connection to {@code address}, or null if no such connection exists. */ 6 RealConnection get(Address address, StreamAllocation streamAllocation) { 7 assert (Thread.holdsLock(this)); 8 for (RealConnection connection : connections) { 9 if (connection.allocations.size() < connection.allocationLimit 10 && address.equals(connection.route().address) 11 && !connection.noNewStreams) { 12 streamAllocation.acquire(connection); 13 return connection; 14 } 15 } 16 return null; 17 }
ConnectionPool连接池是连接的容器,这里用了一个Deque来保存所有的连接RealConnection。而get的过程就是,遍历保存的所有连接来匹配address。同时connection.allocations.size()要满足connection.allocationLimit的限制。 在找到了所需要的连接之后,会acquire该连接。
acquire连接的过程又是什么样的呢?
1public final class StreamAllocation { 2 3...... 4 5 /** 6 * Use this allocation to hold {@code connection}. Each call to this must be paired with a call to 7 * {@link #release} on the same connection. 8 */ 9 public void acquire(RealConnection connection) { 10 assert (Thread.holdsLock(connectionPool)); 11 connection.allocations.add(new StreamAllocationReference(this, callStackTrace)); 12 }
基本上就是给RealConnection的allocations添加一个到该StreamAllocation的引用。这样看来,同一个连接RealConnection似乎同时可以为多个HTTP请求服务。而我们知道,多个HTTP/1.1请求是不能在同一个连接上交叉处理的。那这又是怎么回事呢?
我们来看connection.allocationLimit的更新设置。RealConnection中如下的两个地方会设置这个值:
1public final class RealConnection extends Http2Connection.Listener implements Connection { 2 3...... 4 5 private void establishProtocol(int readTimeout, int writeTimeout, 6 ConnectionSpecSelector connectionSpecSelector) throws IOException { 7 if (route.address().sslSocketFactory() != null) { 8 connectTls(readTimeout, writeTimeout, connectionSpecSelector); 9 } else { 10 protocol = Protocol.HTTP_1_1; 11 socket = rawSocket; 12 } 13 14 if (protocol == Protocol.HTTP_2) { 15 socket.setSoTimeout(0); // Framed connection timeouts are set per-stream. 16 17 Http2Connection http2Connection = new Http2Connection.Builder(true) 18 .socket(socket, route.address().url().host(), source, sink) 19 .listener(this) 20 .build(); 21 http2Connection.start(); 22 23 // Only assign the framed connection once the preface has been sent successfully. 24 this.allocationLimit = http2Connection.maxConcurrentStreams(); 25 this.http2Connection = http2Connection; 26 } else { 27 this.allocationLimit = 1; 28 } 29 } 30 31 /** When settings are received, adjust the allocation limit. */ 32 @Override public void onSettings(Http2Connection connection) { 33 allocationLimit = connection.maxConcurrentStreams(); 34 }
可以看到,若不是HTTP/2的连接,则allocationLimit的值总是1。由此可见,StreamAllocation以及RealConnection的allocations/allocationLimit这样的设计,主要是为了实现HTTP/2 multi stream的特性。否则的话,大概为RealConnection用一个inUse标记就可以了。 那
回到StreamAllocation的findConnection(),来看新创建的RealConnection对象建立连接的过程,即RealConnection的connect():
1public final class RealConnection extends Http2Connection.Listener implements Connection { 2 private final Route route; 3 4 /** The low-level TCP socket. */ 5 private Socket rawSocket; 6 7 /** 8 * The application layer socket. Either an {@link SSLSocket} layered over {@link #rawSocket}, or 9 * {@link #rawSocket} itself if this connection does not use SSL. 10 */ 11 public Socket socket; 12 private Handshake handshake; 13 private Protocol protocol; 14 public volatile Http2Connection http2Connection; 15 public int successCount; 16 public BufferedSource source; 17 public BufferedSink sink; 18 public int allocationLimit; 19 public final List<Reference<StreamAllocation>> allocations = new ArrayList<>(); 20 public boolean noNewStreams; 21 public long idleAtNanos = Long.MAX_VALUE; 22 23 public RealConnection(Route route) { 24 this.route = route; 25 } 26 27 public void connect(int connectTimeout, int readTimeout, int writeTimeout, 28 List<ConnectionSpec> connectionSpecs, boolean connectionRetryEnabled) { 29 if (protocol != null) throw new IllegalStateException("already connected"); 30 31 RouteException routeException = null; 32 ConnectionSpecSelector connectionSpecSelector = new ConnectionSpecSelector(connectionSpecs); 33 34 if (route.address().sslSocketFactory() == null) { 35 if (!connectionSpecs.contains(ConnectionSpec.CLEARTEXT)) { 36 throw new RouteException(new UnknownServiceException( 37 "CLEARTEXT communication not enabled for client")); 38 } 39 String host = route.address().url().host(); 40 if (!Platform.get().isCleartextTrafficPermitted(host)) { 41 throw new RouteException(new UnknownServiceException( 42 "CLEARTEXT communication to " + host + " not permitted by network security policy")); 43 } 44 } 45 46 while (protocol == null) { 47 try { 48 if (route.requiresTunnel()) { 49 buildTunneledConnection(connectTimeout, readTimeout, writeTimeout, 50 connectionSpecSelector); 51 } else { 52 buildConnection(connectTimeout, readTimeout, writeTimeout, connectionSpecSelector); 53 } 54 } catch (IOException e) { 55 closeQuietly(socket); 56 closeQuietly(rawSocket); 57 socket = null; 58 rawSocket = null; 59 source = null; 60 sink = null; 61 handshake = null; 62 protocol = null; 63 64 if (routeException == null) { 65 routeException = new RouteException(e); 66 } else { 67 routeException.addConnectException(e); 68 } 69 70 if (!connectionRetryEnabled || !connectionSpecSelector.connectionFailed(e)) { 71 throw routeException; 72 } 73 } 74 } 75 }
根据路由的类型,来执行不同的创建连接的过程。对于需要创建隧道连接的路由,执行buildTunneledConnection(),而对于普通连接,则执行buildConnection()。
如何判断是否要建立隧道连接呢?来看
1 /** 2 * Returns true if this route tunnels HTTPS through an HTTP proxy. See <a 3 * href="http://www.ietf.org/rfc/rfc2817.txt">RFC 2817, Section 5.2</a>. 4 */ 5 public boolean requiresTunnel() { 6 return address.sslSocketFactory != null && proxy.type() == Proxy.Type.HTTP; 7 }
可以看到,通过代理服务器,来做https请求的连接(http/1.1的https和http2)需要建立隧道连接,而其它的连接则不需要建立隧道连接。
用于建立隧道连接的buildTunneledConnection()的过程:
1 /** 2 * Does all the work to build an HTTPS connection over a proxy tunnel. The catch here is that a 3 * proxy server can issue an auth challenge and then close the connection. 4 */ 5 private void buildTunneledConnection(int connectTimeout, int readTimeout, int writeTimeout, 6 ConnectionSpecSelector connectionSpecSelector) throws IOException { 7 Request tunnelRequest = createTunnelRequest(); 8 HttpUrl url = tunnelRequest.url(); 9 int attemptedConnections = 0; 10 int maxAttempts = 21; 11 while (true) { 12 if (++attemptedConnections > maxAttempts) { 13 throw new ProtocolException("Too many tunnel connections attempted: " + maxAttempts); 14 } 15 16 connectSocket(connectTimeout, readTimeout); 17 tunnelRequest = createTunnel(readTimeout, writeTimeout, tunnelRequest, url); 18 19 if (tunnelRequest == null) break; // Tunnel successfully created. 20 21 // The proxy decided to close the connection after an auth challenge. We need to create a new 22 // connection, but this time with the auth credentials. 23 closeQuietly(rawSocket); 24 rawSocket = null; 25 sink = null; 26 source = null; 27 } 28 29 establishProtocol(readTimeout, writeTimeout, connectionSpecSelector); 30 }
基本上是两个过程:
- 建立隧道连接。
- 建立Protocol。
建立隧道连接的过程,又分为了几个过程:
- 创建隧道请求
- 建立Socket连接
- 发送请求建立隧道
隧道请求是一个常规的HTTP请求,只是请求的内容有点特殊。初始的隧道请求如:
1 /** 2 * Returns a request that creates a TLS tunnel via an HTTP proxy. Everything in the tunnel request 3 * is sent unencrypted to the proxy server, so tunnels include only the minimum set of headers. 4 * This avoids sending potentially sensitive data like HTTP cookies to the proxy unencrypted. 5 */ 6 private Request createTunnelRequest() { 7 return new Request.Builder() 8 .url(route.address().url()) 9 .header("Host", Util.hostHeader(route.address().url(), true)) 10 .header("Proxy-Connection", "Keep-Alive") 11 .header("User-Agent", Version.userAgent()) // For HTTP/1.0 proxies like Squid. 12 .build(); 13 }
建立socket连接的过程如下:
1 private void connectSocket(int connectTimeout, int readTimeout) throws IOException { 2 Proxy proxy = route.proxy(); 3 Address address = route.address(); 4 5 rawSocket = proxy.type() == Proxy.Type.DIRECT || proxy.type() == Proxy.Type.HTTP 6 ? address.socketFactory().createSocket() 7 : new Socket(proxy); 8 9 rawSocket.setSoTimeout(readTimeout); 10 try { 11 Platform.get().connectSocket(rawSocket, route.socketAddress(), connectTimeout); 12 } catch (ConnectException e) { 13 throw new ConnectException("Failed to connect to " + route.socketAddress()); 14 } 15 source = Okio.buffer(Okio.source(rawSocket)); 16 sink = Okio.buffer(Okio.sink(rawSocket)); 17 }
主要是创建一个到代理服务器或HTTP服务器的Socket连接。socketFactory最终来自于OkHttpClient,对于OpenJDK 8而言,默认为DefaultSocketFactory:
1 /** 2 * Returns a copy of the environment's default socket factory. 3 * 4 * @return the default <code>SocketFactory</code> 5 */ 6 public static SocketFactory getDefault() 7 { 8 synchronized (SocketFactory.class) { 9 if (theFactory == null) { 10 // 11 // Different implementations of this method SHOULD 12 // work rather differently. For example, driving 13 // this from a system property, or using a different 14 // implementation than JavaSoft's. 15 // 16 theFactory = new DefaultSocketFactory(); 17 } 18 } 19 20 return theFactory; 21 }
创建隧道的过程是这样子的:
1 /** 2 * To make an HTTPS connection over an HTTP proxy, send an unencrypted CONNECT request to create 3 * the proxy connection. This may need to be retried if the proxy requires authorization. 4 */ 5 private Request createTunnel(int readTimeout, int writeTimeout, Request tunnelRequest, 6 HttpUrl url) throws IOException { 7 // Make an SSL Tunnel on the first message pair of each SSL + proxy connection. 8 String requestLine = "CONNECT " + Util.hostHeader(url, true) + " HTTP/1.1"; 9 while (true) { 10 Http1Codec tunnelConnection = new Http1Codec(null, null, source, sink); 11 source.timeout().timeout(readTimeout, MILLISECONDS); 12 sink.timeout().timeout(writeTimeout, MILLISECONDS); 13 tunnelConnection.writeRequest(tunnelRequest.headers(), requestLine); 14 tunnelConnection.finishRequest(); 15 Response response = tunnelConnection.readResponse().request(tunnelRequest).build(); 16 // The response body from a CONNECT should be empty, but if it is not then we should consume 17 // it before proceeding. 18 long contentLength = HttpHeaders.contentLength(response); 19 if (contentLength == -1L) { 20 contentLength = 0L; 21 } 22 Source body = tunnelConnection.newFixedLengthSource(contentLength); 23 Util.skipAll(body, Integer.MAX_VALUE, TimeUnit.MILLISECONDS); 24 body.close(); 25 26 switch (response.code()) { 27 case HTTP_OK: 28 // Assume the server won't send a TLS ServerHello until we send a TLS ClientHello. If 29 // that happens, then we will have buffered bytes that are needed by the SSLSocket! 30 // This check is imperfect: it doesn't tell us whether a handshake will succeed, just 31 // that it will almost certainly fail because the proxy has sent unexpected data. 32 if (!source.buffer().exhausted() || !sink.buffer().exhausted()) { 33 throw new IOException("TLS tunnel buffered too many bytes!"); 34 } 35 return null; 36 37 case HTTP_PROXY_AUTH: 38 tunnelRequest = route.address().proxyAuthenticator().authenticate(route, response); 39 if (tunnelRequest == null) throw new IOException("Failed to authenticate with proxy"); 40 41 if ("close".equalsIgnoreCase(response.header("Connection"))) { 42 return tunnelRequest; 43 } 44 break; 45 46 default: 47 throw new IOException( 48 "Unexpected response code for CONNECT: " + response.code()); 49 } 50 } 51 }
主要HTTP 的 CONNECT 方法建立隧道。
而建立常规的连接的过程则为:
1 /** Does all the work necessary to build a full HTTP or HTTPS connection on a raw socket. */ 2 private void buildConnection(int connectTimeout, int readTimeout, int writeTimeout, 3 ConnectionSpecSelector connectionSpecSelector) throws IOException { 4 connectSocket(connectTimeout, readTimeout); 5 establishProtocol(readTimeout, writeTimeout, connectionSpecSelector); 6 }
建立socket连接,然后建立Protocol。建立Protocol的过程为:
1 private void establishProtocol(int readTimeout, int writeTimeout, 2 ConnectionSpecSelector connectionSpecSelector) throws IOException { 3 if (route.address().sslSocketFactory() != null) { 4 connectTls(readTimeout, writeTimeout, connectionSpecSelector); 5 } else { 6 protocol = Protocol.HTTP_1_1; 7 socket = rawSocket; 8 } 9 10 if (protocol == Protocol.HTTP_2) { 11 socket.setSoTimeout(0); // Framed connection timeouts are set per-stream. 12 13 Http2Connection http2Connection = new Http2Connection.Builder(true) 14 .socket(socket, route.address().url().host(), source, sink) 15 .listener(this) 16 .build(); 17 http2Connection.start(); 18 19 // Only assign the framed connection once the preface has been sent successfully. 20 this.allocationLimit = http2Connection.maxConcurrentStreams(); 21 this.http2Connection = http2Connection; 22 } else { 23 this.allocationLimit = 1; 24 } 25 }
HTTP/2协议的协商过程在connectTls()的过程中完成。
总结一下OkHttp3的连接RealConnection的含义,或者说是ConnectInterceptor从StreamAllocation中获取的RealConnection对象的状态:
- 对于不使用HTTP代理的HTTP请求,为一个到HTTP服务器的Socket连接。后续直接向该Socket连接中写入常规的HTTP请求,并从中读取常规的HTTP响应。
- 对于不使用代理的https请求,为一个到https服务器的Socket连接,但经过了TLS握手,协议协商等过程。后续直接向该Socket连接中写入常规的请求,并从中读取常规的响应。
- 对于使用HTTP代理的HTTP请求,为一个到HTTP代理服务器的Socket连接。后续直接向该Socket连接中写入常规的HTTP请求,并从中读取常规的HTTP响应。
- 对于使用代理的https请求,为一个到代理服务器的隧道连接,但经过了TLS握手,协议协商等过程。后续直接向该Socket连接中写入常规的请求,并从中读取常规的响应。
关于HTTP代理的更多内容,可以参考HTTP 代理原理及实现(一)。
OkHttp3中对路由的处理大体如此。