In this vignette, we describe how to run a Bayesian network meta-analysis model using this package. First, we’ll need to load the package.
#install.packages("bnma")
#or devtools::install_github("MikeJSeo/bnma")
library(bnma)It is essential to specify the input data in a correct format. We have chosen to use arm-level data with following input variable names: Outcomes, N or SE, Study, and Treat. Outcomes is the trial results. N is the number of respondents used for binary or multinomial model. SE is the standard error used for normal model. Study is the study indicator for the meta analysis. Lastly, Treat is the treatment indicator for each arm. We will use a dataset parkinsons for illustration.
parkinsons
#> $Outcomes
#> [1] -1.22 -1.53 -0.70 -2.40 -0.30 -2.60 -1.20 -0.24 -0.59 -0.73 -0.18 -2.20
#> [13] -2.50 -1.80 -2.10
#>
#> $SE
#> [1] 0.504 0.439 0.282 0.258 0.505 0.510 0.478 0.265 0.354 0.335 0.442 0.197
#> [13] 0.190 0.200 0.250
#>
#> $Treat
#> [1] "Placebo" "Ropinirole" "Placebo" "Pramipexole"
#> [5] "Placebo" "Pramipexole" "Bromocriptine" "Ropinirole"
#> [9] "Bromocriptine" "Ropinirole" "Bromocriptine" "Bromocriptine"
#> [13] "Cabergoline" "Bromocriptine" "Cabergoline"
#>
#> $Study
#> [1] 1 1 2 2 3 3 3 4 4 5 5 6 6 7 7
#>
#> $Treat.order
#> [1] "Placebo" "Pramipexole" "Ropinirole" "Bromocriptine"
#> [5] "Cabergoline"In order to run network meta-analysis in JAGS, we need to relabel study names into to a numeric sequence, i.e. 1 to total number of studies, and relabel the treatment into a numeric sequence according to treatment order specified. If the treatment order is not specified, default is to use the alphabetical order. In the example below, we set placebo as the baseline treatment followed by Pramipexole, Ropinirole, Bromocriptine, and Cabergoline as the treatment order.
network <- with(parkinsons, network.data(Outcomes = Outcomes, Study = Study, Treat = Treat, SE = SE, response = "normal", Treat.order = Treat.order))
network$Treat.order
#> 1 2 3 4 5
#> "Placebo" "Pramipexole" "Ropinirole" "Bromocriptine" "Cabergoline"
network$Study.order
#> 1 2 3 4 5 6 7
#> 1 2 3 4 5 6 7Another important preprocessing step that is done in network.data function is changing the arm-level data into study-level data. We store the study-level data of Outcomes as r, Treat as t, N or SE as n or se. We can see how Outcomes changed into a study-level matrix given below (i.e. row = study). If the Outcomes are multinomial, it will change to a 3 dimensional array.
network$r
#> [,1] [,2] [,3]
#> [1,] -1.22 -1.53 NA
#> [2,] -0.70 -2.40 NA
#> [3,] -0.30 -2.60 -1.2
#> [4,] -0.24 -0.59 NA
#> [5,] -0.73 -0.18 NA
#> [6,] -2.20 -2.50 NA
#> [7,] -1.80 -2.10 NAPriors can be set in the network.data function. If left unspecified, default values are used. For heterogeneity parameters of the random effects model, we follow the data format from a similar Bayesian network meta-analysis R package gemtc. It should be a list of length 3 where the first element should be the distribution (one of dunif, dgamma, dhnorm, dwish) and the next two are the parameters associated with the distribution. Here is an example of assigning a half-normal distribution with mean 0 and standard deviation 5.
network <- with(smoking, network.data(Outcomes = Outcomes, Study = Study, Treat = Treat, N = N, response = "binomial", mean.d = 0.1, hy.prior = list("dhnorm", 0, 5)))Now to run the model, we use the function network.run. The most important parameter is n.run which determines the number of final samples the user wants. Gelman-Rubin statitics is checked automatically every setsize number of iterations and once the series have converged we store the last half of the sequence. If the number of iteration is less than the number of final samples (n.runs), it will sample more to fill the requirement. One of the nice features of this package is that it checks for convergence automatically and will give an error if the sequence has not converged. The parameters tested for convergence are the relative treatment effects, baseline effect, and heterogeneity parameter. The number that is printed during the running of the model is the point estimate of the Gelman-Rubin statistics used to test convergence.
result <- network.run(network, n.run = 30000)
#> Compiling model graph
#> Resolving undeclared variables
#> Allocating nodes
#> Graph information:
#> Observed stochastic nodes: 50
#> Unobserved stochastic nodes: 54
#> Total graph size: 1129
#>
#> Initializing model
#>
#> NOTE: Stopping adaptation
#>
#>
#> [1] 1.002285
#> [1] 1.001515Package includes many summary tools that can be used. One of the useful summary might be the forest plot.
network.forest.plot(result)
# draw.network.graph(network)
# network.autocorr.diag(result)
# network.autocorr.plot(result)
# network.cumrank.tx.plot(result)
# network.deviance.plot(result)
# network.gelman.plot(result)Another nice addition of this package is that multinomial outcome dataset can be analyzed. Here is an example.
network <- with(cardiovascular, network.data(Outcomes, Study, Treat, N, response = "multinomial"))
result <- network.run(network)
#> Compiling model graph
#> Resolving undeclared variables
#> Allocating nodes
#> Graph information:
#> Observed stochastic nodes: 34
#> Unobserved stochastic nodes: 37
#> Total graph size: 1301
#>
#> Initializing model
#>
#> NOTE: Stopping adaptation
#>
#>
#> [1] 1.004925
#> [1] 1.003027
summary(result)
#> $summary.samples
#>
#> Iterations = 1:50000
#> Thinning interval = 1
#> Number of chains = 3
#> Sample size per chain = 50000
#>
#> 1. Empirical mean and standard deviation for each variable,
#> plus standard error of the mean:
#>
#> Mean SD Naive SE Time-series SE
#> d[1,1] 0.000000 0.00000 0.000e+00 0.0000000
#> d[2,1] -0.105781 0.15200 3.925e-04 0.0011348
#> d[3,1] -0.047454 0.11264 2.908e-04 0.0007060
#> d[1,2] 0.000000 0.00000 0.000e+00 0.0000000
#> d[2,2] -0.187928 0.15368 3.968e-04 0.0011633
#> d[3,2] -0.270296 0.11209 2.894e-04 0.0006462
#> sigma[1,1] 0.114651 0.05028 1.298e-04 0.0002421
#> sigma[2,1] 0.004492 0.03434 8.866e-05 0.0001623
#> sigma[1,2] 0.004492 0.03434 8.866e-05 0.0001623
#> sigma[2,2] 0.114464 0.05046 1.303e-04 0.0002280
#>
#> 2. Quantiles for each variable:
#>
#> 2.5% 25% 50% 75% 97.5%
#> d[1,1] 0.00000 0.00000 0.000000 0.000000 0.00000
#> d[2,1] -0.40461 -0.20577 -0.106617 -0.006354 0.19695
#> d[3,1] -0.27155 -0.12048 -0.046886 0.026369 0.17344
#> d[1,2] 0.00000 0.00000 0.000000 0.000000 0.00000
#> d[2,2] -0.49311 -0.28798 -0.186891 -0.086630 0.11355
#> d[3,2] -0.49464 -0.34262 -0.269420 -0.196993 -0.05073
#> sigma[1,1] 0.05188 0.08051 0.103656 0.135924 0.24161
#> sigma[2,1] -0.06343 -0.01431 0.003835 0.022598 0.07599
#> sigma[1,2] -0.06343 -0.01431 0.003835 0.022598 0.07599
#> sigma[2,2] 0.05197 0.08038 0.103510 0.135649 0.24176
#>
#>
#> $Treat.order
#> 1 2 3
#> 1 2 3
#>
#> $deviance
#> Dbar pD DIC
#> 31.14917 60.43631 91.58548
#>
#> $total_n
#> [1] 34
#>
#> attr(,"class")
#> [1] "summary.network.result"We can add continuous or discrete covariates to fit a network meta-regression. If the covariate is continuous, it is centered. Discrete variables need to be 0-1 dummy format. There are three different assumptions for covariate effect: “common”, “independent”, and “exchangeable”.
network <- with(statins, network.data(Outcomes, Study, Treat, N=N, response = "binomial", Treat.order = c("Placebo", "Statin"), covariate = covariate, covariate.type = "discrete", covariate.model = "common"))
result <- network.run(network)
#> Compiling model graph
#> Resolving undeclared variables
#> Allocating nodes
#> Graph information:
#> Observed stochastic nodes: 38
#> Unobserved stochastic nodes: 41
#> Total graph size: 877
#>
#> Initializing model
#>
#> NOTE: Stopping adaptation
#>
#>
#> [1] 1.028327
#> [1] 1.012254
summary(result)
#> $summary.samples
#>
#> Iterations = 1:50000
#> Thinning interval = 1
#> Number of chains = 3
#> Sample size per chain = 50000
#>
#> 1. Empirical mean and standard deviation for each variable,
#> plus standard error of the mean:
#>
#> Mean SD Naive SE Time-series SE
#> beta1[1] 0.00000 0.0000 0.0000000 0.000000
#> beta1[2] -0.29925 0.2630 0.0006790 0.003606
#> d[1] 0.00000 0.0000 0.0000000 0.000000
#> d[2] -0.06834 0.2068 0.0005340 0.002721
#> sd 0.24561 0.1959 0.0005058 0.005107
#>
#> 2. Quantiles for each variable:
#>
#> 2.5% 25% 50% 75% 97.5%
#> beta1[1] 0.00000 0.0000 0.00000 0.00000 0.0000
#> beta1[2] -0.87256 -0.4354 -0.28263 -0.14701 0.1920
#> d[1] 0.00000 0.0000 0.00000 0.00000 0.0000
#> d[2] -0.48066 -0.1826 -0.07248 0.04141 0.3649
#> sd 0.01098 0.1006 0.19903 0.34168 0.7356
#>
#>
#> $Treat.order
#> 1 2
#> "Placebo" "Statin"
#>
#> $deviance
#> Dbar pD DIC
#> 42.58508 25.14313 67.72821
#>
#> $total_n
#> [1] 38
#>
#> attr(,"class")
#> [1] "summary.network.result"Covariate plot shows you how the relative treatment effect changes as the covariate varies.
network.covariate.plot(result, base.treatment = "Placebo", comparison.treatment = "Statin")
Another useful addition of this network package is the ability to model baseline risk. We can have “common”, “independent”, or “exchangeable” assumption on the baseline slopes and “independenet” and “exchangeable” assumption on the baseline risk. Here we demonstrate a common baseline slope and exchangeable baseline risk model.
network <- with(certolizumab, network.data(Outcomes = Outcomes, Treat = Treat, Study = Study, N = N, response = "binomial", Treat.order = Treat.order, baseline = "common", baseline.risk = "exchangeable"))
result <- network.run(network)
#> Compiling model graph
#> Resolving undeclared variables
#> Allocating nodes
#> Graph information:
#> Observed stochastic nodes: 24
#> Unobserved stochastic nodes: 34
#> Total graph size: 670
#>
#> Initializing model
#>
#> NOTE: Stopping adaptation
#>
#>
#> [1] 1.025623
#> [1] 1.006675
summary(result)
#> $summary.samples
#>
#> Iterations = 1:50000
#> Thinning interval = 1
#> Number of chains = 3
#> Sample size per chain = 50000
#>
#> 1. Empirical mean and standard deviation for each variable,
#> plus standard error of the mean:
#>
#> Mean SD Naive SE Time-series SE
#> B -0.7634 0.2748 0.0007095 0.004566
#> b_bl[1] 0.0000 0.0000 0.0000000 0.000000
#> b_bl[2] -0.7634 0.2748 0.0007095 0.004566
#> b_bl[3] -0.7634 0.2748 0.0007095 0.004566
#> b_bl[4] -0.7634 0.2748 0.0007095 0.004566
#> b_bl[5] -0.7634 0.2748 0.0007095 0.004566
#> b_bl[6] -0.7634 0.2748 0.0007095 0.004566
#> b_bl[7] -0.7634 0.2748 0.0007095 0.004566
#> d[1] 0.0000 0.0000 0.0000000 0.000000
#> d[2] 1.8685 0.2245 0.0005798 0.001932
#> d[3] 2.1308 0.2070 0.0005346 0.002297
#> d[4] 2.0284 0.4242 0.0010952 0.006118
#> d[5] 1.6265 0.1994 0.0005149 0.002190
#> d[6] 0.3214 0.5843 0.0015086 0.010205
#> d[7] 2.1405 0.2801 0.0007232 0.002958
#> sd 0.1842 0.1690 0.0004364 0.003426
#>
#> 2. Quantiles for each variable:
#>
#> 2.5% 25% 50% 75% 97.5%
#> B -1.258646 -0.91803 -0.7803 -0.6281 -0.1867
#> b_bl[1] 0.000000 0.00000 0.0000 0.0000 0.0000
#> b_bl[2] -1.258646 -0.91803 -0.7803 -0.6281 -0.1867
#> b_bl[3] -1.258646 -0.91803 -0.7803 -0.6281 -0.1867
#> b_bl[4] -1.258646 -0.91803 -0.7803 -0.6281 -0.1867
#> b_bl[5] -1.258646 -0.91803 -0.7803 -0.6281 -0.1867
#> b_bl[6] -1.258646 -0.91803 -0.7803 -0.6281 -0.1867
#> b_bl[7] -1.258646 -0.91803 -0.7803 -0.6281 -0.1867
#> d[1] 0.000000 0.00000 0.0000 0.0000 0.0000
#> d[2] 1.420613 1.75563 1.8670 1.9815 2.3185
#> d[3] 1.738296 2.01326 2.1242 2.2406 2.5630
#> d[4] 1.194794 1.78232 2.0228 2.2762 2.8743
#> d[5] 1.238935 1.51839 1.6249 1.7334 2.0285
#> d[6] -0.914115 -0.03493 0.3473 0.7126 1.3834
#> d[7] 1.599047 1.98896 2.1406 2.2902 2.7012
#> sd 0.005032 0.06699 0.1439 0.2507 0.6027
#>
#>
#> $Treat.order
#> 1 2 3 4 5
#> "Placebo" "CZP" "Adalimumab" "Etanercept" "Infliximab"
#> 6 7
#> "Rituximab" "Tocilizumab"
#>
#> $deviance
#> Dbar pD DIC
#> 27.81316 18.55810 46.37126
#>
#> $total_n
#> [1] 24
#>
#> attr(,"class")
#> [1] "summary.network.result"We included another inconsistency model that can be used to test consistency assumption. Here we can specify a pair where we want to nodesplit and test the inconsistency assumptions. For instance if we specify treatment pair = c(3, 9), we are finding the difference in the direct and indirect evidence of treatment 3 and 9. Inconsistency estimate and the corresponding p-value are reported in the summary. If the p-value is small, it means that we can reject the null hypothesis that direct and indirect evidence agree. We can repeat for all the pairs in the network and identify pairs that might be inconsistent. Refer to Dias et al. 2010 (i.e. Checking consistency in mixed treatment comparison meta-analysis) for more details.
network <- with(thrombolytic, nodesplit.network.data(Outcomes, Study, Treat, N, response = "binomial", pair = c(3,9), type = "fixed"))
result <- nodesplit.network.run(network)
#> Compiling model graph
#> Resolving undeclared variables
#> Allocating nodes
#> Graph information:
#> Observed stochastic nodes: 102
#> Unobserved stochastic nodes: 59
#> Total graph size: 2263
#>
#> Initializing model
#>
#> NOTE: Stopping adaptation
#>
#>
#> [1] 1.001145
#> [1] 1.000863
summary(result)
#> $summary.samples
#>
#> Iterations = 1:50000
#> Thinning interval = 1
#> Number of chains = 3
#> Sample size per chain = 50000
#>
#> 1. Empirical mean and standard deviation for each variable,
#> plus standard error of the mean:
#>
#> Mean SD Naive SE Time-series SE
#> d[1] 0.000000 0.00000 0.000e+00 0.000e+00
#> d[2] -0.001639 0.03028 7.818e-05 1.913e-04
#> d[3] -0.160129 0.04331 1.118e-04 3.545e-04
#> d[4] -0.044119 0.04637 1.197e-04 2.370e-04
#> d[5] -0.112600 0.05980 1.544e-04 4.608e-04
#> d[6] -0.154461 0.07701 1.988e-04 5.725e-04
#> d[7] -0.464456 0.10055 2.596e-04 5.625e-04
#> d[8] -0.196424 0.21888 5.652e-04 1.259e-03
#> d[9] 0.003543 0.03679 9.498e-05 2.211e-04
#> diff 1.240178 0.42021 1.085e-03 3.972e-03
#> direct 1.403850 0.41631 1.075e-03 3.953e-03
#> oneminusprob 0.000620 0.02489 6.427e-05 9.511e-05
#> prob 0.999380 0.02489 6.427e-05 9.511e-05
#>
#> 2. Quantiles for each variable:
#>
#> 2.5% 25% 50% 75% 97.5%
#> d[1] 0.00000 0.00000 0.000000 0.00000 0.000000
#> d[2] -0.06086 -0.02212 -0.001711 0.01890 0.057709
#> d[3] -0.24522 -0.18937 -0.159940 -0.13082 -0.075455
#> d[4] -0.13560 -0.07539 -0.043938 -0.01264 0.045951
#> d[5] -0.22989 -0.15294 -0.112784 -0.07227 0.004793
#> d[6] -0.30499 -0.20673 -0.154431 -0.10220 -0.003386
#> d[7] -0.66049 -0.53275 -0.464416 -0.39616 -0.267566
#> d[8] -0.62674 -0.34411 -0.196064 -0.04925 0.230792
#> d[9] -0.06874 -0.02116 0.003506 0.02838 0.075439
#> diff 0.45256 0.95197 1.226160 1.51547 2.106371
#> direct 0.62462 1.11818 1.389193 1.67643 2.262463
#> oneminusprob 0.00000 0.00000 0.000000 0.00000 0.000000
#> prob 1.00000 1.00000 1.000000 1.00000 1.000000
#>
#>
#> $deviance
#> NULL
#>
#> $`Inconsistency estimate`
#> [1] 1.240178
#>
#> $p_value
#> [1] 0.00124
#>
#> attr(,"class")
#> [1] "summary.nodesplit.network.result"Default summary measure when analyzing binary outcomes is the odds ratio. We have added an option to calculate risk difference, relative risk, and number needed to treat by incorporating external baseline risk in treatment A (i.e. placebo)
#Using metaprop function from meta package, we meta-analyze placebo event proportion.
#library(meta)
#placebo_index <- which(certolizumab$Treat == "Placebo")
#meta.pla <- metaprop(event = certolizumab$Outcomes[placebo_index], n = certolizumab$N[placebo_index], method = "GLMM", sm = "PLOGIT")
#mean.A = meta.pla$TE.random; prec.A = 1/meta.pla$tau^2
network <- with(certolizumab, network.data(Outcomes = Outcomes, Treat = Treat, Study = Study, N = N, response = "binomial", mean.A = -2.27, prec.A = 2.53))
result <- network.run(network)
#> Compiling model graph
#> Resolving undeclared variables
#> Allocating nodes
#> Graph information:
#> Observed stochastic nodes: 24
#> Unobserved stochastic nodes: 32
#> Total graph size: 653
#>
#> Initializing model
#>
#> NOTE: Stopping adaptation
#>
#>
#> [1] 1.006228
#> [1] 1.045021
summary(result, extra.pars = c("RD", "RR", "NNT"))
#> $summary.samples
#>
#> Iterations = 1:50000
#> Thinning interval = 1
#> Number of chains = 3
#> Sample size per chain = 50000
#>
#> 1. Empirical mean and standard deviation for each variable,
#> plus standard error of the mean:
#>
#> Mean SD Naive SE Time-series SE
#> NNT[2] 1.209e+02 4.002e+04 1.033e+02 1.033e+02
#> NNT[3] 4.815e+00 4.395e+03 1.135e+01 1.134e+01
#> NNT[4] 2.212e+03 8.725e+05 2.253e+03 2.253e+03
#> NNT[5] -1.591e+01 1.329e+02 3.431e-01 3.433e-01
#> NNT[6] -1.701e+01 8.051e+02 2.079e+00 1.988e+00
#> NNT[7] -4.439e+00 5.283e+03 1.364e+01 1.364e+01
#> RD[2] 6.925e-02 1.455e-01 3.756e-04 9.392e-04
#> RD[3] 2.602e-01 2.790e-01 7.204e-04 2.356e-03
#> RD[4] 8.452e-03 1.290e-01 3.331e-04 1.174e-03
#> RD[5] -8.588e-02 5.602e-02 1.446e-04 2.426e-04
#> RD[6] -6.707e-02 9.953e-02 2.570e-04 6.115e-04
#> RD[7] -1.140e-02 1.076e-01 2.777e-04 6.981e-04
#> RR[2] 1.760e+00 1.793e+00 4.629e-03 1.129e-02
#> RR[3] 4.040e+00 4.093e+00 1.057e-02 3.010e-02
#> RR[4] 1.137e+00 1.569e+00 4.052e-03 1.373e-02
#> RR[5] 1.948e-01 2.976e-01 7.683e-04 2.206e-03
#> RR[6] 3.921e-01 1.065e+00 2.750e-03 6.855e-03
#> RR[7] 9.269e-01 1.263e+00 3.261e-03 8.168e-03
#> d[1] 0.000e+00 0.000e+00 0.000e+00 0.000e+00
#> d[2] 3.441e-01 1.119e+00 2.889e-03 6.671e-03
#> d[3] 1.476e+00 1.905e+00 4.919e-03 1.534e-02
#> d[4] -2.404e-01 1.073e+00 2.770e-03 8.461e-03
#> d[5] -2.000e+00 7.131e-01 1.841e-03 5.401e-03
#> d[6] -1.998e+00 1.539e+00 3.975e-03 9.658e-03
#> d[7] -5.006e-01 1.118e+00 2.886e-03 6.701e-03
#> sd 9.450e-01 6.694e-01 1.728e-03 1.066e-02
#>
#> 2. Quantiles for each variable:
#>
#> 2.5% 25% 50% 75% 97.5%
#> NNT[2] -2.555e+02 -17.33566 6.38468 21.467419 247.68880
#> NNT[3] -1.002e+02 1.29031 2.65443 7.470110 96.51590
#> NNT[4] -2.532e+02 -35.33880 -16.25608 6.198035 212.26733
#> NNT[5] -4.514e+01 -19.31588 -12.88850 -8.776045 -4.24975
#> NNT[6] -6.535e+01 -20.44507 -12.87220 -8.136118 24.61807
#> NNT[7] -2.519e+02 -34.77728 -16.64671 2.542706 213.48427
#> RD[2] -1.023e-01 -0.01314 0.02883 0.105423 0.49776
#> RD[3] -9.561e-02 0.02713 0.18146 0.460817 0.85309
#> RD[4] -1.289e-01 -0.04882 -0.02093 0.016881 0.41124
#> RD[5] -2.140e-01 -0.11198 -0.07651 -0.050776 -0.01876
#> RD[6] -2.146e-01 -0.10651 -0.06912 -0.041085 0.13211
#> RD[7] -1.466e-01 -0.05757 -0.02723 0.002731 0.27991
#> RR[2] 1.652e-01 0.83924 1.33247 2.080001 6.17805
#> RR[3] 1.544e-01 1.32225 2.86065 5.395071 14.76767
#> RR[4] 1.393e-01 0.47275 0.72648 1.193601 4.99909
#> RR[5] 3.457e-02 0.10624 0.15111 0.210669 0.59047
#> RR[6] 7.071e-03 0.06634 0.15114 0.335862 2.35128
#> RR[7] 6.674e-02 0.38597 0.65310 1.033843 3.70908
#> d[1] 0.000e+00 0.00000 0.00000 0.000000 0.00000
#> d[2] -1.895e+00 -0.19518 0.33014 0.877003 2.62927
#> d[3] -1.961e+00 0.31994 1.34002 2.511708 5.64004
#> d[4] -2.068e+00 -0.81148 -0.35326 0.202013 2.26013
#> d[5] -3.470e+00 -2.34127 -1.98895 -1.654086 -0.57790
#> d[6] -5.066e+00 -2.82195 -1.99104 -1.170583 1.03862
#> d[7] -2.814e+00 -1.02358 -0.46919 0.037487 1.71495
#> sd 1.553e-01 0.51096 0.78231 1.181343 2.77938
#>
#>
#> $Treat.order
#> 1 2 3 4 5
#> "Adalimumab" "CZP" "Etanercept" "Infliximab" "Placebo"
#> 6 7
#> "Rituximab" "Tocilizumab"
#>
#> $deviance
#> Dbar pD DIC
#> 26.98476 22.65087 49.63563
#>
#> $total_n
#> [1] 24
#>
#> attr(,"class")
#> [1] "summary.network.result"