TMLAI-10008.pdf

Page 1 of 7

Transactions on Machine Learning and Artificial Intelligence - Vol. 9, No. 2

Publication Date: April, 25, 2021

DOI:10.14738/tmlai.92.10008. Chen, W. W. S. (2021). Conversion Among Arma Models and State-Space Representation. Transactions on Machine Learning and

Artificial Intelligence, 9(2). 53-59.

Services for Science and Education – United Kingdom

Conversion Among Arma Models and State-Space Representation

William W. S. Chen

Department of Statistics, The George Washington University

Washington D.C. 20013

ABSTRACT

We present the ARMA models (or Non-Markovian) and state-space (or Markovian)

representation relationship. Then we break the problem into three different cases

to discuss how one form could be converted to another form. In case A, we assume

that we know the state-space representation then we convert it into the ARMA

model. In case B, we reverse the situation, given the ARMA model we convert into

state-space representation. In Case C, we combine the first two cases, conversion

the two forms in either directions.

KEY WORDS: ARMA Model, Conversion to another form, Markovian Representation,

replace t by t+1, replace t+3 by t, State-Space representation, Time Index.

INTRODUCTION

In discussing linear systems it is often more convenient to use the state-space (or Markovian)

representation of the relationship between input and output rather than the explicit form. The

state-space form gives a very compact description which is valid provided the relationship

between the input and output can be expressed in terms of a finite order linear difference

equation. The basic idea rests on the well-known result that any finite order linear difference

equation can be expressed as a vector first order equation. For this reason, state-space

representation had a profound impact on time series analysis and many related areas. Davis

and Vinter(1985) applied these techniques in control of linear systems. Later, Hannan and

Deistler (1988) used it in linear system of statistical theory. This is a rich class for time series

models and going well beyond the linear ARMA models. In econometrics the structural time

series models developed by Harvey (1990) formulated like the classical decomposition model

directly in terms of components of interest such as trend, noise. The objective of current paper

is not creating a new theory for time series. We wish to build a bridge between ARMA model

and state-space representation.

RELATIONSHIP BETWEEN STATE-SPACE AND ARMA MODEL

Let be a univariate ARMA(p,q) time series with p>q

We define then

for

t x

11 11 ...... .... t t p t p t t q tq xx xaa a = ++ + f f qq - --- - --

| ,1 ( | ....) t it t i t t x Ex xx + + = -

t it t | x x + = i £ 0

| ,1 a Ea xx a t it t i t t t i + + = = ( | .....) i 0 - + £

Page 2 of 7

Transactions on Machine Learning and Artificial Intelligence (TMLAI) Vol 9, Issue 2, April - 2021

Services for Science and Education – United Kingdom

The state-space formulation is

where is a length p column vector and F is a pxp matrix.

This representation is based on writing

F G

The matrices F and G are identified by the following way.

The infinite MA representation for is

Finally, we have the following results.

And expectation given , ...is

| 0 for i>0 t i t a + =

t+1 t 1 z z = + F Gat+

2| 1 1|

3| 1 2|

| 1 1|

. .

t t

t t tt

tt tt

t t

t pt t p t

x x

z a

x x

++ +

++ + -

éù éù é ù éù êú êú ê ú êú

== +

ëû ëû ë û ëû

t i x +

t+i 1 1 1 1 = .......

ti k tik

ti i t k tik

k i

x x

aa a a

Y YY

+ + - =

+ - - + + - =

= å

++ + å

ti k tik |t

k i

x a Y¥

+ + - =

= å

| 1 1 1

11 | = (2.1)

ti i t k tik t

k i

i t t it

xaa

a x

Y Y

+ + - + + - =

- + +

= + å

1 1 1| for i=1 (2.2) t t tt xax +++ = +

11 11 ...... .... t p t p p t t p t p q t pq x x xa a a + + = ++ + f fq q - + + - -- - + -

t x t 1 x -

| 1 1| | xx x t pt t p t p tt + + = ++ + f f - ...... 0 since p-q>0

|1 1 1 | (2.3) t pt p t t pt x ax + + = + y - + +

Page 3 of 7

Chen, W. W. S. (2021). Conversion Among Arma Models and State-Space Representation. Transactions on Machine Learning and Artificial

Intelligence, 9(2). 53-59.

URL: http://dx.doi.org/10.14738/tmlai.92.10008.

Summarize (2.1), (2.2), and (2.3), we can write the state-space formulation as below:

The bottom row of the F matrix gives AR coefficients and the G vector gives the first p-1

coefficients, which can be used to calculates the coefficients

CASE STUDY

Case A. Given the following State-Space representation for a bivariate time series, write it in

ARMA Form.

From (3a.1), rewrite as

Substitute (3a.4) into (3a.2)

Rewrite (3a.5) in the form

2| 1 1| 1

3| 1 2|

| 1 1 1 | 1| 1

0 1 0.. 0 1

0 0 1.. 0

. . ... . .

. . . ... . . .

. 0 . ... 1 . .

t t

t t tt

tt tt

t pt p p t p t p

x x

FF F Y

++ +

+ + - + - -

é ù é ùé ù é ù

ê ú ê úê ú ê ú

= +

ë û ë ûë û ë û

() ()

( ) imply ( ) ( )

( ) ( ) so (B) (B)= (B) ( )

t t

t t tt

Bx Ba

B x a x Ba

B where B

F Q

Q Y

Q Y YF Q

= =

1| 1 |

1, 1

2| 1 21 22 23 1| 1 2

2, 1

1| 1 33 |

0 1 0 10

0 0 01

t t tt

X X

X f f f X gg

Y fY

+ +

++ +

+ +

æ ö æö æ ö æö ç ÷ ç÷ æ ö ç ÷ ç÷ = + ç ÷ è ø è ø èø è ø èø

1| 1 1| 1, 1

2| 1 21 | 22 1| 23 | 1 1, 1 2 2, 1

t+1|t+1 33 | 2, 1

(3a.1)

(3a.2)

tt tt t

t t tt t t tt t t

tt t

X Xa

X f X f X f Y ga g a

fy a

++ + +

++ + + +

= +

= + ++ +

= + (3a.3)

1| 1 1, 1 (3a.4) X Xa tt t t + ++ = -

2 1, 2 21 22 1 1, 1 23 1 1, 1 2 2, 1 ( ) (3a.5) X a f X f X a f Y ga g a t t t tt t t t + + ++ + + - = + - ++ +

2 2

22 21 2 23 2 1 22 1, 2 2 2, 2 (1 ) (1 ( ) ) (3 .6) tt tt -- - f B f B X f B Y g f B a g Ba a ++ ++ = + - +

Page 4 of 7

Transactions on Machine Learning and Artificial Intelligence (TMLAI) Vol 9, Issue 2, April - 2021

Services for Science and Education – United Kingdom

From (3a.3), we replace t by t+1 and have

Rewrite above equation in the form

Combine (3a.6) and (3a.7), we write in bivariate time series form,

where

Case B. Write the following bivariate ARMA (1,0,1) model in the State-Space formulation.

From (3b.1), we replace t by t+1 and consider the conditional expectation up to time t

Same reason for equation (3b.2)

If we replace t by t+2 then consider the conditional expectation up to t, we have

y t+2 33 1 2, 2 t t = + fy a + +

33 t+2 2, 2 (1- B)y (3a.7) t f a = +

2 2

22 21 23 2 1 22 2 1, 2

33 2 2, 2

1 1( )

0 1 0 1

t t

fB fB fB x g f B gB a

f B y a

+ +

æ ö -- - æ ö æ ö + - æ ö ç ÷ç ÷ = ç ÷ç ÷ è ø - è ø è øè ø

( ) ( ) 2 2 1, 2

12 1

2 2, 2

t t

x a

IBB IB

y a

FF Q + +

+ +

æ ö æ ö - - ç ÷ = - ç ÷ è ø è ø

22 21 23

1 2

1 22 2

1 0 0 , , 01 0 0 0

( )

0 0

f f f I

gf g

F F

æö æ ö æ ö == = ç÷ ç ÷ ç ÷ èø è ø è ø

æ ö -- - = ç ÷ è ø

11 12 11 12 1

21 22 21 22 2

( ) t t

t t

X a

I BI B

Y a

ff qq

æö æö æö æö æ ö - ç÷ ç÷ ç÷ ç÷ = ç ÷ -

èø èø èø èø è ø

11 12 11 12 1

21 22 21 22 2

1 1

t t

BB BB X a

BB BB Y a

ff qq

æ öæ ö -- -- æö æö æ ö

ç ÷ç ÷ ç÷ ç÷ = ç ÷ è øè ø -- -- èø èø è ø

11 1 12 1 1, 11 1, 1 12 2, 1 (3b.1) X X Ya a a t t tt t t = ++ ff q q -- - - - -

22 1 21 1 2, 22 2, 1 21 1, 1 (3b.2) Y Y Xa a a tt tt t t =+ + ff q q -- - - - -

1| 11 12 11 1, 12 2, (3b.3) X X Ya a tt t t t t + = + f fq q - -

1| 22 21 22 2, 21 1, (3b.4) Y YX a a tt t t t t + = + ff q q - -

2| 11 1| 12 1| (3b.5) X XY t t tt tt + ++ = + f f

2| 22 1| 21 1| (3b.6) YYX t t tt tt ++ + = + f f

Page 5 of 7

Chen, W. W. S. (2021). Conversion Among Arma Models and State-Space Representation. Transactions on Machine Learning and Artificial

Intelligence, 9(2). 53-59.

URL: http://dx.doi.org/10.14738/tmlai.92.10008.

Both of the (3b.5) and (3b.6) are linear combination of the (3b.3) and (3b.4). Hence it should

not show in State-space formula. From (3b.1) and (3b.2), we have the Following results.

Substitute (3b.7) and (3b.8) into (3b.3) and (3b.4), replace t by t+1 then we can derive

Combine (3b.7), (3b.8), (3b.9) and (3b.10), we could formulate State-Space model as follow:

Case C. Write the ARMA (2,0,2) model in the state-space formulation then convert it back to

ARMA (2,0,2) model.

Spell out (3c.2), we have

From (3c.3) and (3c.4), we replace t by t+1 and we derive

1 1| 1, 1

1 1| 2, 1

(3b.7)

(3b.8)

t tt t

XX a

YY a

++ +

= +

2| 1 11 1| 1, 1 12 1| 2, 1 11 1, 1 12 2, 1 ( ) ( ) X X a Ya a a t t tt t tt t t t ++ + + + + + + = ++ + f f qq - -

11 1| 12 1| 11 11 1, 1 12 12 2, 1 ( ) ( ) (3b.9) = ++ f f fq fq XY a a tt tt t t ++ + + - + -

2| 1 22 1| 2, 1 21 1| 1, 1 22 2, 1 21 1, 1 ( ) ( ) Y Ya X a a a t t tt t tt t t t ++ + + + + + + = ++ + f f qq - -

22 1| 21 1| 22 22 2, 1 21 21 1, 1 ( ) ( ) (3b.10) =+ + f f fq fq YX a a tt tt t t ++ + + - + -

1| 1

1| 1 1, 1

2| 1 1| 11 12 11 11 12 12 2, 1

2| 1 1| 21 22 21 21 22 22

00 1 0 1 0

00 0 1 0 1

0 0

tt t

tt t t

t t tt t

t t tt

X X

Y Y a

X X a

Y Y

ff fq fq

ff fqfq

+ +

+ + +

++ + +

++ +

æ ö æö æ öæ ö ç ÷ ç÷ ç ÷ç ÷æ ö

= + ç ÷ - - è ø

è øè ø - - è ø èø

2 2

12 12

21 2

12 12

(1 ) (1 )

(1 ) (1 ) (3c.1)

t t

t t k tk

B BZ B Ba

Z B B B Ba a

ff qq

ff qq y ¥ -

- =

- - = - -

= - - - - = å

state-space representation

1| 1 1| 1

2| 1 2| 1 1

3| 1 2 1| 1 2| 2 1

(3c.3)

(3c.4)

+ (3c.5)

tt tt t

tt tt t t t

Z Za

ZZ a

Z ZZ a

f fy

++ + +

++ + + +

= +

2| 1 2| 2 2

3| 1 3| 2 1 2

(3c.6)

(3c.7)

tt tt t

ZZ a

ZZ a y

++ ++ +

= -

1| 1

2| 1 1| 1 1

3| 1 2 1 2| 2

01 0 1

0 0 1 (3c.2)

tt t

t t tt t

tt tt

Z Z

Z Za

Z Z

ff y

+ +

++ + +

++ +

æ ö æö æ ö æö ç ÷ ç÷ ç ÷ ç÷ = +

è ø èø è ø èø

Page 6 of 7

Transactions on Machine Learning and Artificial Intelligence (TMLAI) Vol 9, Issue 2, April - 2021

Services for Science and Education – United Kingdom

Substitute (3c.8), (3c.3) and (3c.4) into (3c.5), we derive

Lastly, we replace t+3 by t in (3c.9) and rearrange order

At last, we can write (3c.10) in the ARMA (2,0,2) model

In above equation we had applied the relationship among It needs some

algebraic process to derive these results. We leave it in the appendix.

CONCLUDING REMARKS

In control theory of linear systems or in econometrics, these two forms played the central key

roles. In this paper, we present a bridge to link the ARMA models and state space representation

in such a way even a researcher does not familiar with time series theories can switch one form

to the other. So that he can plug in the needed form into SAS package to complete the computing.

We had chosen our notation consistent with the SAS package to minimize the possible

confusion. It seems much more straightforward to convert from ARMA model to state-space

representation. However, the process needs more careful on two tricky factors. We must be

certain that all the variables are independent to each other. Secondly, the time index must be

carefully treated.

References

[1] Brockwell, P.J. and Davis, R.A. (1995) A first course in time series analysis. Springer-Verlag, New York, Inc.

Washington Statistical Society workshop, October,30-31, 1995.

[2] Davis, M.H.A. and Vinter, R.B. (1985) Stochastic Modelling and Control, Chapman and Hall, London.

[3] Hannan, E.J. and Deistler, M. (1988) The Statistical Theory of Linear System, John Wiley, New York.

[4] Hannan, E.J. and Rissanen, J.(1982) Recursive estimation of mixed autoregressive moving average order.

Biometrika,69,81-94.

[5] Harvey, A.C. and Fernandes, C. (1989) Time Series Models for count data of qualitative observations. J.

Business and Economic Statistics, 407-422.

[6] Harvey, A.C. (1990) Forecasting, Structural Time Series Models and the Kalman Filter, Cambridge University

Press, Cambridge.

[7] Harvey, A.C. (1981) Time Series Models. Halsted Press, New York.

[8] Wei, W.W.S. (2006) Time Series Analysis Univariate and Multivariate Methods. Second Edition, Boston

Pearson Addison Wesley.

3| 1 3| 3 3 1 2

substitute (3c.6) to right side of (3c.7)

(3c.8) ZZ aa tt tt t t ++ ++ + + = - -y

3 3 12 2 1 1 1 2 2 11 21 ( ) ( - - )+ (3c.9) Z a a Z a Za a a t t t t t tt t t + + + + + ++ + + =+ + y f f yy - +

11 2 2 2 1 1 1 12 2 2

2 2 1 1 1 1 1 2 11 2 2

( ) ( - - )+

= ( - ) ( ) (3c.10)

tt t t t tt t t

t tt t t

Za a Z a Za a a

Z Za a a

yf f yy

f f y f y fy f

- - - -- - -

-- - -

=+ + - +

+ ++ + - -

2 2

12 12 (1 ) (1 ) - - ff qq B BZ B Ba t t = - -

1 1 1 2 2 11 2 where - ( - ) - ( ) q y f q y fy f = = - -

i i , and . qy f i

Page 7 of 7

Chen, W. W. S. (2021). Conversion Among Arma Models and State-Space Representation. Transactions on Machine Learning and Artificial

Intelligence, 9(2). 53-59.

URL: http://dx.doi.org/10.14738/tmlai.92.10008.

APPENDIX

21 2

12 12

(1 ) (1 ) t t k tk

Z B B B Ba a ff qq y ¥ -

- =

= - - - - = å

2 2 23

12 12 1 2 3

1 1 2 11 2

(1 ) (1 )(1 ......)

(1 ( ) ( ) ...)

t t

B Ba B B B B B a

B Ba

qq ff yy y

y f y yf f

- - = - - + + + +

= + - + - - +

1 1 1 2 11 2 2 ( ) , ( ) y f q y fy f q - = - - - = -