{"id":2983,"date":"2020-12-31T07:09:00","date_gmt":"2020-12-30T22:09:00","guid":{"rendered":"http:\/\/itoshi.main.jp\/tech\/?page_id=2983"},"modified":"2021-01-02T09:32:35","modified_gmt":"2021-01-02T00:32:35","slug":"corrido","status":"publish","type":"page","link":"http:\/\/itoshi.main.jp\/tech\/0100-rl\/corrido\/","title":{"rendered":"10 q, qnet, gru\/lstm \u306e\u30d1\u30d5\u30a9\u30fc\u30de\u30f3\u30b9\u30c1\u30a7\u30c3\u30af"},"content":{"rendered":"

q, qnet, gru, lstm \u306e\u30b3\u30fc\u30c7\u30a3\u30f3\u30b0\u306b\u9593\u9055\u3044\u306f\u306a\u3044\u306e\u3067\u3057\u3087\u3046\u304b\uff1f\u305d\u306e\u78ba\u8a8d\u3092\u3059\u308b\u305f\u3081\u306b\u306f\u3001Q\u5024\u304c\u6b63\u3057\u3044\u5024\u306b\u53ce\u675f\u3059\u308b\u304b<\/strong>\u3092\u30c1\u30a7\u30c3\u30af\u3059\u308b\u3053\u3068\u304c\u5927\u5207\u3067\u3059\u3002<\/p>\n

\u307e\u305f\u3001\u77ed\u671f\u8a18\u61b6\u30e6\u30cb\u30c3\u30c8(gru\/lstm)\u306e\u77ed\u671f\u8a18\u61b6\u306e\u80fd\u529b\u306f\u3069\u308c\u304f\u3089\u3044\u306a\u306e\u3067\u3057\u3087\u3046\u304b\u3002\u5b9a\u91cf\u7684\u306b\u77e5\u308a\u305f\u3044\u3067\u3059\u3002<\/p>\n

\u3053\u306e\u3088\u3046\u306a\u3053\u3068\u3092\u8abf\u3079\u308b\u305f\u3081\u306b\u306f\u3001\u30bf\u30b9\u30af\u306f\u3067\u304d\u308b\u3060\u3051\u30b7\u30f3\u30d7\u30eb\u306a\u65b9\u304c\u3088\u3044\u3067\u3057\u3087\u3046\u3002sawamptour \u306f\u304b\u306a\u308a\u30b7\u30f3\u30d7\u30eb\u3067\u3059\u304c\u307e\u3060Q\u5024\u3092\u8abf\u3079\u308b\u306b\u306f\u72b6\u614b\u6570\u304c\u591a\u304f\u8907\u96d1\u3067\u3059\u3002\u305d\u3053\u3067\u3001\u72b6\u614b\u6570\u304c\u7a76\u6975\u306b\u5c11\u306a\u3044\u300c\u5eca\u4e0b\u30bf\u30b9\u30af\u300d<\/strong>\u3092\u8003\u3048\u307e\u3057\u305f\u3002sim_corridor.py \u3067\u3059\u3002<\/p>\n

\u3053\u306e\u7ae0\u3067\u306f\u3001\u3053\u306e\u5eca\u4e0b\u30bf\u30b9\u30af\u3092\u4f7f\u3063\u3066\u5404\u30a2\u30eb\u30b4\u30ea\u30ba\u30e0\u306e\u30d1\u30d5\u30a9\u30fc\u30de\u30f3\u30b9\u3092\u30c1\u30a7\u30c3\u30af\u3057\u307e\u3059\u3002lstm\u3068gru\u306b\u3064\u3044\u3066\u306f\u3001\u3053\u306e\u4e21\u8005\u306e\u30d1\u30d5\u30a9\u30fc\u30de\u30f3\u30b9\u306f\u985e\u4f3c\u3057\u3066\u3044\u307e\u3057\u305f\u304c\u3001\u611f\u899a\u7684\u306b\u30d1\u30d5\u30a9\u30fc\u30de\u30f3\u30b9\u304c\u3088\u3044\u3068\u601d\u308f\u308c\u308bgru\u306e\u307f\u3092\u30c1\u30a7\u30c3\u30af\u3057\u307e\u3057\u305f\u3002<\/p>\n

sim_corridor \u306e\u4f7f\u3044\u65b9<\/h2>\n

\u4f7f\u3044\u65b9\u306f\u3001sim_swamptour \u3068\u540c\u3058\u3067\u3059\u3002<\/p>\n

[code]
\n> python sim_corridor.py
\n[\/code]
\n\u3067\u3001\u5b9f\u884c\u65b9\u6cd5\u304c\u8868\u793a\u3055\u308c\u307e\u3059\u3002<\/p>\n

[code]
\n—- \u4f7f\u3044\u65b9 —————————————
\n3\u3064\u306e\u30d1\u30e9\u30e1\u30fc\u30bf\u3092\u6307\u5b9a\u3057\u3066\u5b9f\u884c\u3057\u307e\u3059<\/p>\n

> python sim_corridor.py [agt_type] [task_type] [process_type]<\/p>\n

[agt_type] :q, qnet, lstm, gru
\n[task_type] : L8g23v, L8g23, L8g34, L8g67, L8g34567
\n[process_type] :learn\/L, more\/M, graph\/G, anime\/A
\n\u4f8b > python sim_corridor.py q L8g23v L
\n—————————————————
\n[\/code]<\/p>\n

\u30bf\u30b9\u30af\u306f\u3001L8g23v, L8g23, L8g34, L8g67, L8g34567, \u306e5\u7a2e\u985e\u304c\u3059\u3050\u306b\u4f7f\u3048\u307e\u3059\u3002<\/p>\n

\u5168\u3066\u306e\u30bf\u30b9\u30af\u306f\u30011\u6b21\u5143\u306e\u30d5\u30a3\u30fc\u30eb\u30c9\u3067\u30d7\u30ec\u30a4\u3057\u307e\u3059\u3002\u30b4\u30fc\u30eb\u306e\u4f4d\u7f6e\uff08\u9752\uff09\u3092\u899a\u3048\u3066\u3001\u305d\u3053\u307e\u3067\u79fb\u52d5\u3057[\u53f3\u306b\u79fb\u52d5]\u3057\u3001[\u8131\u51fa]\u3059\u308b\u3068\u5831\u916c\u304c\u3082\u3089\u3048\u307e\u3059\u3002L8g23v \u4ee5\u5916\u306f\u3001\u30b4\u30fc\u30eb\u306e\u4f4d\u7f6e\u304c\u8868\u793a\u3055\u308c\u308b\u306e\u306f\u958b\u59cb\u6642\u306e\u307f\u306a\u306e\u3067\u3001\u4f4d\u7f6e\u3092\u899a\u3048\u306a\u304f\u3066\u306f\u306a\u308a\u307e\u305b\u3093\u3002\u30bf\u30b9\u30af\u306b\u3088\u3063\u3066\u30b4\u30fc\u30eb\u306e\u51fa\u73fe\u4f4d\u7f6e\u304c\u5909\u308f\u308a\u307e\u3059\u3002<\/p>\n

\"\"<\/a><\/p>\n

\u4ee5\u4e0b\u306f\u3001L8g23v \u3092q \u306b\u5b66\u7fd2\u3055\u305b\u305f\u3068\u304d\u306e\u30a2\u30cb\u30e1\u30fc\u30b7\u30e7\u30f3\u3067\u3059\u3002\u3046\u307e\u304f\u9752\u306e\u30b4\u30fc\u30eb\u3067\u8131\u51fa\u3067\u304d\u3066\u3044\u308b\u3053\u3068\u304c\u5206\u304b\u308a\u307e\u3059\u3002<\/p>\n

\"\"<\/a><\/p>\n

\u4ee5\u4e0b\u306f\u3001L8g23 \u3092gru \u306b\u5b66\u7fd2\u3055\u305b\u305f\u3068\u304d\u306e\u30a2\u30cb\u30e1\u30fc\u30b7\u30e7\u30f3\u3067\u3059\u3002\u30b4\u30fc\u30eb\u306f\u9014\u4e2d\u3067\u6d88\u3048\u3066\u3057\u307e\u3044\u307e\u3059\u304c\u3001\u30a8\u30fc\u30b8\u30a7\u30f3\u30c8\u306f\u305d\u306e\u5834\u6240\u3092\u899a\u3048\u3066\u8131\u51fa\u3057\u3066\u3044\u308b\u3053\u3068\u304c\u5206\u304b\u308a\u307e\u3059\u3002<\/p>\n

\"\"<\/a><\/p>\n

sim_corridor.py \u3068\u540c\u69d8\u3001\u4ee5\u4e0b\u306e(1)\u3068(2)\u3067\u3001env_corridor.py \u306b\u65b0\u3057\u3044\u30bf\u30b9\u30af\u3092\u8ffd\u52a0\u3059\u308b\u3053\u3068\u3082\u3067\u304d\u307e\u3059\u3002<\/p>\n

(1) env_coridor.py \u306e\u5217\u6319\u578bclass TaskType \u306b mytask \u3092\u8ffd\u52a0\uff08\u540d\u524d\u306f\u4efb\u610f\uff09\u3002<\/p>\n

\r\nclass TaskType(Enum):\r\n    \"\"\"\r\n    \u30bf\u30b9\u30af\u30bf\u30a4\u30d7\u306e\u5217\u6319\u578b\r\n    \"\"\"\r\n    L8g23v = auto()\r\n    L8g23 = auto()\r\n    L8g34 = auto()\r\n    L8g67 = auto()\r\n    L8g34567 = auto()\r\n    # mytask = auto() # \u30aa\u30ea\u30b8\u30ca\u30eb\u30bf\u30b9\u30af\u3092\u8ffd\u52a0\r\n<\/pre>\n
(2) mytask \u306e\u30d1\u30e9\u30e1\u30fc\u30bf\u3092\u3001class Env \u306e set_task_type() \u306b\u8ffd\u52a0\u3002<\/p>\n
\r\nclass Env(core.coreEnv):\r\n    def __init__():\r\n        --- \u7701\u7565 \r\n        elif task_type == TaskType.L8g34567:\r\n            self.field_length = 8\r\n            self.goal_candidate = (3, 4, 5, 6, 7)\r\n            self.pos_start = 0\r\n            self.reward_fail = 0\r\n            self.reward_move = 0\r\n            self.reward_goal = 1\r\n            self.step_until_goal_hidden = 1\r\n\r\n\u3000\u3000\u3000\u3000# \u4ee5\u4e0b\u306e\u3088\u3046\u306b\u8ffd\u52a0\r\n            \"\"\"\r\n        elif task_type == TaskType.mytask:  # mytask \u306e\u8a2d\u5b9a\u8ffd\u52a0\r\n            self.field_length = 20\r\n            self.goal_candidate = list(range(10, 20))\r\n            self.pos_start = 0\r\n            self.reward_fail = 0\r\n            self.reward_move = 0\r\n            self.reward_goal = 1\r\n            self.step_until_goal_hidden = 1\r\n            \"\"\"\r\n<\/pre>\n
\u30b7\u30df\u30e5\u30ec\u30fc\u30b7\u30e7\u30f3\u306e\u30d1\u30e9\u30e1\u30fc\u30bf\uff08\u4e00\u56de\u3067\u884c\u3046\u30b9\u30c6\u30c3\u30d7\u6570 N_STEP\u3001\u7d42\u4e86\u6761\u4ef6 EARY_STOP_STEP \u7b49\uff09\u3082\u30c7\u30d5\u30a9\u30eb\u30c8\u5024\u4ee5\u5916\u3067\u8a2d\u5b9a\u3057\u305f\u5834\u5408\u306b\u306f\u3001sim_corridor.py \u306e\u4ee5\u4e0b\u306e\u30b3\u30fc\u30c9\u306e\u5f8c\u306b\u3001myTask\u306e\u8a2d\u5b9a\u3092\u8ffd\u52a0\u3057\u307e\u3059\u3002<\/p>\n
\r\n--- \u7701\u7565 \r\n    elif task_type == TaskType.L8g34567:\r\n        N_STEP = 5000\r\n        SHOW_Q_INTERVAL =200\r\n        EARY_STOP_STEP = None\r\n        EARY_STOP_REWARD = 1\r\n        AGT_EPSILON = 0.4\r\n        AGT_ANIME_EPSILON = 0.0\r\n\r\n    # \u4ee5\u4e0b\u306e\u3088\u3046\u306b\u8ffd\u52a0\r\n        \"\"\"\r\n    elif task_type == TaskType.mytask:  # mytask\u306e\u30b7\u30df\u30e5\u30ec\u30fc\u30b7\u30e7\u30f3\u30d1\u30e9\u30e1\u30fc\u30bf\u8ffd\u52a0\r\n        N_STEP = 5000\r\n        SHOW_Q_INTERVAL =200\r\n        EARY_STOP_STEP = None\r\n        EARY_STOP_REWARD = 1\r\n        AGT_EPSILON = 0.4\r\n        AGT_ANIME_EPSILON = 0.0\r\n        \"\"\"\r\n<\/pre>\n
Q\u5024\u306e\u7406\u8ad6\u5024<\/h2>\n
L8g23v <\/strong>\u306eQ\u306e\u7406\u8ad6\u5024\u3092\u8003\u3048\u3066\u307f\u307e\u3059\u3002[math]\\gamma=0.9[\/math]\u3092\u60f3\u5b9a\u3057\u307e\u3059\u3002\u307e\u305a\u3001\u30b4\u30fc\u30eb\u306e\u4f4d\u7f6e\u304c2\u306b\u51fa\u305f\u6642\u306e\u30a8\u30d4\u30bd\u30fc\u30c9\u3092\u8003\u3048\u307e\u3059\u3002\u5fa9\u7fd2\u3067\u3059\u304c\u3001Q\u5024 [math]Q(x, a)[\/math] \u306f\u3001\u89b3\u6e2c[math]x[\/math]\u3092\u53d7\u3051\u53d6\u3063\u3066\u884c\u52d5[math]a[\/math]\u3092\u51fa\u529b\u3057\u305f\u5f8c\u304b\u3089\u53d7\u3051\u53d6\u308b\u5831\u916c\u306e\u548c\u3067\u3059\u3002\u5831\u916c\u306e\u548c\u306f\u3001\u5272\u5f15\u5831\u916c\u7387\u3092\u4e57\u7b97\u3057\u305f\u305d\u306e\u30a8\u30d4\u30bd\u30fc\u30c9\u5185\u3067\u306e\u548c\u3068\u306a\u308a\u307e\u3059\u3002
\n

\n[math]
\n\\begin{eqnarray}
\nQ(x(t), a(t)) & \\approx & r(t)  \\\\
\n& & + \\gamma r(t+1) \\\\
\n& & + \\gamma^2 r(t+2) + \\cdots
\n\\end{eqnarray}
\n[\/math]
\n<\/center><\/p>\n
\u3053\u308c\u306f\u6b21\u306ex(t + 1)\u306eQ\u5024\u3092\u4f7f\u3063\u3066\u4ee5\u4e0b\u306e\u3088\u3046\u306b\u8868\u3059\u3053\u3068\u304c\u3067\u304d\u307e\u3059\u3002<\/p>\n

\n[math]
\n\\begin{eqnarray}
\nQ(x(t), a(t)) & \\approx & r(t)  \\\\
\n & & + \\gamma \\max_{a’} Q(x(t+1), a’)
\n\\end{eqnarray}
\n[\/math]
\n<\/center><\/p>\n
\u3067\u306f\u3001L8g23v \u3092\u8003\u3048\u307e\u3057\u3087\u3046\u3002\u4e0b\u306e\u56f3\u306e\u3088\u3046\u306b\u3001[math]t=0, 1, 2[\/math] \u3067\u30ed\u30dc\u30c3\u30c8\u304c\u4f4d\u7f6e2\u306e\u3068\u3053\u308d\u307e\u3067\u9032\u3093\u3060\u3053\u3068\u3092\u60f3\u5b9a\u3057\u307e\u3059\u3002<\/p>\n
\"\"<\/a><\/p>\n
t=2 \u306e\u89b3\u6e2cx\u306e\u6642\u306b\u300c\u8131\u51fa\u300d\u3092\u9078\u3076\u3068\u5831\u916c 1 \u3092\u53d7\u3051\u53d6\u3063\u3066\u7d42\u4e86\u3057\u307e\u3059\u306e\u3067 Q(x, \u8131\u51fa)=1 \u3067\u3059\u3002\u300c\u53f3\u3078\u300d\u3092\u9078\u3093\u3060\u5834\u5408\u306b\u306f\u3001\u3053\u308c\u4ee5\u964d\u5831\u916c\u306f\u51fa\u306a\u3044\u306e\u3067 Q(x, \u53f3\u3078)=0 \u3068\u306a\u308a\u307e\u3059\u3002<\/p>\n
\u3053\u306e\uff11\u30b9\u30c6\u30c3\u30d7\u524d\u306e t=1 \u306e\u89b3\u6e2cx\u3067\u306f\u3001\u300c\u8131\u51fa\u300d\u3092\u9078\u3076\u3068 Q(x, \u8131\u51fa)=0 \u3067\u3059\u3002\u300c\u53f3\u3078\u300d\u3092\u9078\u3093\u3060\u5834\u5408\u306b\u306f\u3001\u56f3\u306b\u793a\u3057\u305f\u3088\u3046\u306b\u3001Q(x, \u53f3\u3078)= [math]r(t=1) + \\gamma r(t=2) [\/math]\u3068\u306a\u308a\u30010.9 \u3068\u306a\u308a\u307e\u3059\u3002<\/p>\n
\u3053\u306e\u3088\u3046\u306b\u8003\u3048\u3066\u3001\uff12\u30b9\u30c6\u30c3\u30d7\u524d\u3067\u306f\u3001Q(x, \u8131\u51fa)=0\u3067\u3001Q(x, \u53f3\u3078)=0.81 \u3068\u306a\u308a\u307e\u3059\u3002<\/p>\n
\u30b4\u30fc\u30eb\u306e\u4f4d\u7f6e\u304c3\u3060\u3063\u305f\u3068\u304d\u3082\u3053\u306e\u3088\u3046\u306b\u8003\u3048\u3066\u3001\u4ee5\u4e0b\u306e\u3088\u3046\u306b\u6570\u5024\u304c\u6c42\u307e\u308a\u307e\u3059\u3002<\/p>\n
\"\"<\/a><\/p>\n
\u3067\u306f\u3001L8g23\u3067\u306e\u7406\u8ad6\u5024<\/strong>\u306f\u3069\u3046\u306a\u308b\u3067\u3057\u3087\u3046\u304b\u3002L8g23\u3067\u306f\u30b4\u30fc\u30eb\u304c\u898b\u3048\u306a\u304f\u306a\u308a\u307e\u3059\u304c\u3001\u30a8\u30fc\u30b8\u30a7\u30f3\u30c8\u304c\u30b4\u30fc\u30eb\u306e\u4f4d\u7f6e\u3092\u533a\u5225\u3059\u308b\u3053\u3068\u304c\u3067\u304d\u308b\u306a\u3089\u3001Q\u5024\u306e\u7406\u8ad6\u5024\u306fL8g23v \u3068\u540c\u3058\u306b\u306a\u308b<\/strong>\u306f\u305a\u3067\u3059\u3002<\/p>\n
Q\u5024\u3092\u5b66\u7fd2\u3067\u304d\u308b\u304b L8g23v <\/h2>\n
q \u306e\u30d1\u30d5\u30a9\u30fc\u30de\u30f3\u30b9<\/h3>\n
\u305d\u308c\u3067\u306fQ\u306e\u53ce\u675f\u3092\u898b\u3066\u307f\u307e\u3059\u3002
\n[code]
\n(mRL) python sim_corridor.py q L8g23v L
\n[\/code]
\n\u3067L8g23v \u306e\u30b7\u30df\u30e5\u30ec\u30fc\u30b7\u30e7\u30f3\u3092\u958b\u59cb\u3057\u307e\u3059\u3002<\/p>\n
\u30b7\u30df\u30e5\u30ec\u30fc\u30b7\u30e7\u30f3\u306e\u9593\u3001\u4ee5\u4e0b\u306e\u3088\u3046\u306bQ\u5024\u304c\u8868\u793a\u3055\u308c\u307e\u3059\u3002<\/p>\n
\u5de6\u306e[\u3000]\u306e\u4e2d\u304cobservation \u3092\u8868\u3057\u3066\u304a\u308a\u3001\u305d\u306e\u4e2d\u306e”1″\u306f\u3001\u30ed\u30dc\u30c3\u30c8\u306e\u4f4d\u7f6e\u3092\u793a\u3057\u307e\u3059\u3002”3″\u304c\u30b4\u30fc\u30eb\u3001”0″\u304c\u4f55\u3082\u306a\u3044\u30bb\u30eb\u3067\u3059\u3002\u53f3\u5074\u304cQ(0)\u3068Q(1)\u306e\u5024\u3067\u3059\u3002\u77e2\u5370\u306f\u5927\u304d\u3044\u65b9\u3092\u6307\u3057\u3066\u3044\u307e\u3059\u3002<\/p>\n
Q\u306e\u5024\u3092\u898b\u308b\u3068\u3001\u5b8c\u5168\u306b\u7406\u8ad6\u5024\u3068\u540c\u3058\u5024\u306b\u53ce\u675f\u3057\u3066\u3044\u308b\u3053\u3068\u304c\u5206\u304b\u308a\u307e\u3059\u3002Q\u5b66\u7fd2\u306e\u30a2\u30eb\u30b4\u30ea\u30ba\u30e0\u306f\u3088\u3055\u305d\u3046\u3067\u3059\u306d\u3002<\/p>\n
[code]
\nobservation       : Q(0), Q(1)
\n[1 0 3 0 0 0 0 0] :  0.00 –>  0.81  # (0.00, 0.81)
\n[0 1 3 0 0 0 0 0] :  0.00 –>  0.90  # (0.00, 0.90)
\n[0 0 1 0 0 0 0 0] :  1.00 <–  0.00  # (1.00, 0.00)
\n[0 0 3 1 0 0 0 0] :  0.00 –>  0.00  # (0.00, 0.00)<\/p>\n
[1 0 0 3 0 0 0 0] :  0.00 –>  0.73  # (0.00, 0.73)
\n[0 1 0 3 0 0 0 0] :  0.00 –>  0.81  # (0.00, 0.81)
\n[0 0 1 3 0 0 0 0] :  0.00 –>  0.90  # (0.00, 0.90)
\n[0 0 0 1 0 0 0 0] :  1.00 <–  0.00  # (1.00, 0.00)
\n[\/code]<\/p>\n
qnet \u306e\u30d1\u30d5\u30a9\u30fc\u30de\u30f3\u30b9<\/h3>\n
\u305d\u308c\u3067\u306fqnet \u306f\u3069\u3046\u3067\u3057\u3087\u3046\u304b\u3002<\/p>\n
[code]
\n(mRL) python sim_corridor.py qnet L8g23v L
\n[\/code]
\n\u3067L8g23v \u306e\u30b7\u30df\u30e5\u30ec\u30fc\u30b7\u30e7\u30f3\u3092\u958b\u59cb\u3057\u307e\u3059\u3002<\/p>\n
[code]
\nobservation       : Q(0), Q(1)
\n[1 0 3 0 0 0 0 0] : -0.00 –>  0.81  # (0.00, 0.81)
\n[0 1 3 0 0 0 0 0] : -0.00 –>  0.91  # (0.00, 0.90)
\n[0 0 1 0 0 0 0 0] :  1.00 <–  0.01  # (1.00, 0.00)
\n[0 0 3 1 0 0 0 0] : -0.00 –>  0.01  # (0.00, 0.00)<\/p>\n
[1 0 0 3 0 0 0 0] : -0.01 –>  0.73  # (0.00, 0.73)
\n[0 1 0 3 0 0 0 0] : -0.00 –>  0.81  # (0.00, 0.81)
\n[0 0 1 3 0 0 0 0] :  0.00 –>  0.90  # (0.00, 0.90)
\n[0 0 0 1 0 0 0 0] :  1.00 <–  0.02  # (1.00, 0.00)
\n[\/code]
\n# \u306f\u6b63\u3057\u3044\u5024\u3067\u3059\u3002\u6bd4\u8f03\u306e\u305f\u3081\u306b\u8a18\u8f09\u3057\u3066\u3044\u307e\u3059\u3002<\/p>\n
0.01\u7a0b\u5ea6\u306e\u8aa4\u5dee\u306f\u898b\u3089\u308c\u307e\u3059\u304c\u3001\u6b63\u3057\u3044Q\u5024\u306b\u53ce\u675f\u3057\u3066\u3044\u308b\u3068\u8a00\u3063\u3066\u3088\u3044\u3067\u3057\u3087\u3046\u3002<\/p>\n
\u30cb\u30e5\u30fc\u30e9\u30eb\u30cd\u30c3\u30c8\u3092\u4f7f\u3063\u3066\u3044\u308b\u306e\u3067\u3001\u5b66\u7fd2\u306b\u3088\u3063\u3066\u30c7\u30fc\u30bf\u306b\u306a\u3044\u5165\u529b\u306b\u5bfe\u3059\u308b\u51fa\u529b\u3082\u5909\u5316\u3057\u307e\u3059\u3002\u305d\u306e\u5f71\u97ff\u304c\u51fa\u3066\u3044\u308b\u306e\u3067\u306f\u306a\u3044\u304b\u3068\u601d\u3044\u307e\u3059\u3002q\u306eQ-table \u3067\u306f\u305d\u306e\u3088\u3046\u306a\u5f71\u97ff\u306f\u3042\u308a\u307e\u305b\u3093\u3067\u3057\u305f\u3002<\/p>\n
gru \u306e\u30d1\u30d5\u30a9\u30fc\u30de\u30f3\u30b9<\/h3>\n
gru\u3068lstm \u306f\u307b\u307c\u540c\u3058\u306a\u306e\u3067\u3001\u3053\u3053\u3067\u306f gru\u306e\u7d50\u679c\u3092\u898b\u3066\u3044\u304d\u305f\u3044\u3068\u601d\u3044\u307e\u3059\u3002<\/p>\n
[code]
\nobservation       : Q(0), Q(1)
\n[1 0 3 0 0 0 0 0] : -0.00 –>  0.81  # (0.00, 0.81)
\n[0 1 3 0 0 0 0 0] : -0.09 –>  0.86  # (0.00, 0.90)
\n[0 0 1 0 0 0 0 0] :  0.92 <–  0.01  # (1.00, 0.00)
\n[0 0 3 1 0 0 0 0] : -0.00 <– -0.06  # (0.00, 0.00)<\/p>\n
[1 0 0 3 0 0 0 0] :  0.01 –>  0.68  # (0.00, 0.73)
\n[0 1 0 3 0 0 0 0] : -0.05 –>  0.80  # (0.00, 0.81)
\n[0 0 1 3 0 0 0 0] :  0.01 –>  0.92  # (0.00, 0.90)
\n[0 0 0 1 0 0 0 0] :  0.97 <– -0.24  # (1.00, 0.00)
\n[\/code]
\n# \u306f\u6b63\u3057\u3044\u5024\u3067\u3059\u3002\u6bd4\u8f03\u306e\u305f\u3081\u306b\u8a18\u8f09\u3057\u3066\u3044\u307e\u3059\u3002<\/p>\n
\u5b8c\u5168\u306b\u306f\u4e00\u81f4\u3057\u3066\u3044\u307e\u305b\u3093\u304c\u3001\u305d\u308c\u3089\u3057\u3044\u5024\u306b\u306f\u306a\u3063\u3066\u3044\u307e\u3059\u3002qnet \u306b\u6bd4\u3079\u308b\u3068\u3001\u904e\u53bb\u306e\u5c65\u6b74\u306b\u3082\u5f71\u97ff\u3055\u308c\u308b\u305b\u3044\u304b\u3001\u82e5\u5e72\u4e0d\u5b89\u5b9a\u306a\u3068\u3053\u308d\u304c\u3042\u308b\u3088\u3046\u3067\u3059\u3002<\/p>\n
\u77ed\u671f\u8a18\u61b6\u306e\u30c6\u30b9\u30c8 L8g23<\/h2>\n
L8g23 \u306f\u3001\u30b4\u30fc\u30eb\u306e\u4f4d\u7f6e\u304c\u521d\u3081\u306b\u3057\u304b\u63d0\u793a\u3055\u308c\u307e\u305b\u3093\u3002\u305d\u306e\u305f\u3081\u540c\u3058\u89b3\u6e2c\u5024\u306b\u5bfe\u3057\u3066\u3082\u3001\u63d0\u793a\u3055\u308c\u305f\u30b4\u30fc\u30eb\u306e\u4f4d\u7f6e\u306b\u3088\u3063\u3066\u884c\u52d5\u3092\u5909\u3048\u308b\u5fc5\u8981\u304c\u3042\u308a\u307e\u3059\u3002<\/p>\n
q \u306e\u30d1\u30d5\u30a9\u30fc\u30de\u30f3\u30b9<\/h3>\n
\u307e\u305a\u3001q \u3067\u3053\u306e\u30bf\u30b9\u30af\u3092\u8a66\u3057\u305f\u3068\u304d\u306eQ\u5024\u3067\u3059\u3002q\u306f\u77ed\u671f\u8a18\u61b6\u306e\u6a5f\u80fd\u304c\u306a\u3044\u306e\u3067\u3046\u307e\u304f\u3067\u304d\u307e\u305b\u3093\u3002<\/p>\n
[code]
\n   observation       : Q(0),      Q(1)
\n\u4f4d\u7f6e0 1 2 3 4 5 6 7
\n A0[1 0 3 0 0 0 0 0] :  0.00 –>  0.37  # (0.00, 0.81)
\n B0[0 1 0 0 0 0 0 0] :  0.00 –>  0.45  # (0.00, 0.90)
\n C0[0 0 1 0 0 0 0 0] :  0.59 <–  0.40  # (1.00, 0.00)
\n D0[0 0 0 1 0 0 0 0] :  0.57 <–  0.00  # (0.00, 0.00)<\/p>\n
 A1[1 0 0 3 0 0 0 0] :  0.00 –>  0.37  # (0.00, 0.73)
\n B1[0 1 0 0 0 0 0 0] :  0.00 –>  0.45  # (0.00, 0.81)
\n C1[0 0 1 0 0 0 0 0] :  0.59 <–  0.40  # (0.00, 0.90)
\n D1[0 0 0 1 0 0 0 0] :  0.57 <–  0.00  # (1.00, 0.00)
\n[\/code]
\n\u4f8b\u3048\u3070\u3001C0\u3068C1\u306f\u3001\u540c\u3058\u89b3\u6e2c\u3067\u3059\u304cC0\u306f2\u306e\u4f4d\u7f6e\u306b\u30b4\u30fc\u30eb\u304c\u63d0\u793a\u3055\u308c\u305f\u5f8c\u3067\u3042\u308a\u3001C1\u306f3\u306e\u4f4d\u7f6e\u306b\u30b4\u30fc\u30eb\u304c\u63d0\u793a\u3055\u308c\u305f\u5f8c\u306a\u306e\u3067C0\u306e\u6642\u306b\u306f\u3001\u300c\u8131\u51fa\u300d\u3092\u9078\u3076\u3068\u5831\u916c\u304c1\u3067\u3042\u308a\u3001C1\u306e\u6642\u306b\u306f\u300c\u8131\u51fa\u300d\u3092\u9078\u3093\u3067\u3057\u307e\u3046\u3068\u5831\u916c\u304c0\u306b\u306a\u308a\u307e\u3059\u3002\u3057\u304b\u3057\u3001C0\u3068C1\u306eQ\u5024\u306f\u540c\u3058\u5024<\/strong>\u3068\u306a\u3063\u3066\u3044\u307e\u3059\u3002<\/p>\n
\u540c\u69d8\u306b\u3001B0\u3068B1\u3001D0\u3068D1\u3067\u3082\u89b3\u6e2c\u304c\u540c\u3058\u5024\u3067\u3059\u306e\u3067Q\u5024\u3082\u540c\u3058\u306b\u306a\u308a\u307e\u3059\u3002A0\u3068A1\u306e\u89b3\u6e2c\u5024\u306f\u30b4\u30fc\u30eb\u304c\u898b\u3048\u3066\u3044\u308b\u306e\u3067\u9055\u3046\u306e\u3067\u3059\u304c\u3001\u305d\u3053\u304b\u3089\u5148\u306e\u672a\u6765\u304c\u540c\u3058\u306b\u306a\u308b\u306e\u3067\u3001\u540c\u3058Q\u5024\u306b\u53ce\u675f\u3057\u3066\u3044\u307e\u3059\u3002<\/p>\n
gru \u306e\u30d1\u30d5\u30a9\u30fc\u30de\u30f3\u30b9<\/h3>\n
\u305d\u308c\u3067\u306f\u3001\u77ed\u671f\u8a18\u61b6\u30e6\u30cb\u30c3\u30c8\u3092\u4f7f\u3063\u3066\u3044\u308bgru<\/strong> \u306f\u3069\u3046\u3067\u3057\u3087\u3046\u304b\u300215000\u30b9\u30c6\u30c3\u30d7\u5b66\u7fd2\u3092\u884c\u3063\u305f\u5f8c\u306eQ\u5024\u3067\u3059\u3002<\/p>\n
[code]
\nobservation          : Q(0),      Q(1)
\n\u4f4d\u7f6e0 1 2 3 4 5 6 7
\n A0[1 0 3 0 0 0 0 0] : -0.01 –>  0.80  # (0.00, 0.81)
\n B0[0 1 0 0 0 0 0 0] : -0.00 –>  0.92  # (0.00, 0.90)
\n C0[0 0 1 0 0 0 0 0] :  1.02 <– -0.05  # (1.00, 0.00)
\n D0[0 0 0 1 0 0 0 0] : -0.00 –>  0.01  # (0.00, 0.00)<\/p>\n
 A1[1 0 0 3 0 0 0 0] : -0.08 –>  0.71  # (0.00, 0.73)
\n B1[0 1 0 0 0 0 0 0] :  0.02 –>  0.82  # (0.00, 0.81)
\n C1[0 0 1 0 0 0 0 0] :  0.00 –>  0.85  # (0.00, 0.90)
\n D1[0 0 0 1 0 0 0 0] :  1.02 <– -0.07  # (1.00, 0.00)
\n[\/code]
\n\u540c\u3058\u89b3\u6e2c\u60c5\u5831\u3067\u3042\u308bB0\u3068B1, C0\u3068C1, D0\u3068D1\u3067\u7570\u306a\u308bQ\u5024\u304c\u51fa\u529b\u3055\u308c\u3066\u3044\u307e\u3059\u3002\u305d\u3057\u3066\u3001\u305d\u308c\u3089\u306e\u5024\u306f\u3001\u304b\u306a\u308a\u7406\u8ad6\u5024\u306b\u8fd1\u3044\u5024\u306b\u53ce\u675f<\/strong>\u3057\u3066\u3044\u308b\u3053\u3068\u304c\u5206\u304b\u308a\u307e\u3059\u3002gru \u304c\u60f3\u5b9a\u3057\u3066\u3044\u305f\u3088\u3046\u306b\u6a5f\u80fd<\/strong>\u3057\u3066\u3044\u308b\u3068\u8a00\u3048\u308b\u3067\u3057\u3087\u3046\u3002<\/p>\n
gru \u306e\u9650\u754c\u3001L8g34, L8g67<\/h2>\n
L8g23\u3092\u5b66\u7fd2\u3059\u308b\u3053\u3068\u304c\u3067\u304d\u305fgru \u3067\u3057\u305f\u304c\u3001L8g34, L8g67 \u3067\u306f\u3069\u3046\u306a\u306e\u3067\u3057\u3087\u3046\u304b\uff1fL8g23\u3068\u6bd4\u3079\u3066\u3001\u8a18\u61b6\u3092\u4fdd\u6301\u3059\u308b\u6642\u9593\u304c\u5c11\u3057\u9577\u304f\u306a\u308b\u306e\u3067\u96e3\u6613\u5ea6\u306f\u5c11\u3057\u4e0a\u304c\u308b\u306f\u305a\u3067\u3059\u3002<\/p>\n
\u5b9f\u969b\u306b\u3084\u3063\u3066\u307f\u308b\u3068\u3001\u3067\u304d\u305f\u308a\u3067\u304d\u306a\u304b\u3063\u305f\u308a\u3068\u3001\u306f\u3063\u304d\u308a\u3057\u306a\u3044\u7d50\u679c\u306b\u306a\u308a\u307e\u3059\u3002<\/p>\n
\u305d\u3053\u3067\u3001\u5168\u3066\u306e\u6761\u4ef6\u306710000\u30b9\u30c6\u30c3\u30d7\u306e\u5b66\u7fd2\u309250\u56de\u884c\u3044\u3001\u5b66\u7fd2\u304c\u6210\u529f\u3059\u308b\u78ba\u7387\u3092\u30b9\u30c6\u30c3\u30d7\u6bce\u306b\u30d7\u30ed\u30c3\u30c8\u3057\u307e\u3057\u305f\uff08\u3053\u306e\u30d7\u30ed\u30c3\u30c8\u306fmemoryRL\u306b\u542b\u307e\u308c\u3066\u3044\u307e\u305b\u3093\u3002\u307e\u305f\u3001L8g45, L8g56\u306f\u30bf\u30b9\u30af\u306b\u542b\u307e\u308c\u3066\u3044\u307e\u305b\u3093\u3002\u72ec\u81ea\u306e\u5225\u306a\u30d7\u30ed\u30b0\u30e9\u30e0\u3067\u8a08\u7b97\u3057\u307e\u3057\u305f\uff09\u3002<\/p>\n
\"\"<\/a><\/p>\n
\u305d\u306e\u7d50\u679c\u3001L8g23\u3067\u306f\u6210\u529f\u3059\u308b\u78ba\u7387\u306f0.8\u4ee5\u4e0a\u3067\u3042\u308b\u306e\u306b\u5bfe\u3057\u3066\u30011\u30b9\u30c6\u30c3\u30d7\u3060\u3051\u8a18\u61b6\u3059\u308b\u6642\u9593\u304c\u5897\u3048\u305fL8g34\u3067\u306f\u3001\u6210\u529f\u78ba\u7387\u304c0.5\u307e\u3067\u4e0b\u304c\u3063\u3066\u3057\u307e\u3046\u3053\u3068\u304c\u5206\u304b\u308a\u307e\u3057\u305f\u3002\u3055\u3089\u306b\u3001L8g45\u3060\u30680.1\u7a0b\u5ea6\u306b\u306a\u3063\u3066\u3057\u307e\u3044\u3001L8g56, L8g57 \u3060\u3068\u6210\u529f\u78ba\u7387\u306f\u307b\u307c0\u3068\u306a\u3063\u3066\u3057\u307e\u3044\u307e\u3057\u305f\u3002<\/p>\n
\u3053\u306e\u6642\u306e\u30b7\u30df\u30e5\u30ec\u30fc\u30b7\u30e7\u30f3\u306f\u3001\u5165\u529b\u5c64\u306e\u6b21\u306e\u4e2d\u9593\u5c64\u306f20\u500b\u306eLeLU\u30e6\u30cb\u30c3\u30c8\u3001\u6b21\u306e\u5c64\u306fgru\u30e6\u30cb\u30c3\u30c820\u500b\u3001\u305d\u3057\u3066\u6b21\u304c\u51fa\u529b\u3068\u306a\u3063\u3066\u3044\u307e\u3057\u305f\u3002<\/p>\n
\u3053\u306e\u6570\u3092\u591a\u304f\u3057\u305f\u3089\u6210\u529f\u78ba\u7387\u304c\u4e0a\u304c\u308b\u304b\u3082\u3057\u308c\u306a\u3044\u3068L8g45\u3067\u30e6\u30cb\u30c3\u30c8\u3092\u591a\u304f\u3057\u305f\u6761\u4ef6\u3067\u8a66\u3057\u307e\u3057\u305f\u304c\u3001\u6210\u529f\u78ba\u7387\u306f\u4e0a\u304c\u308a\u307e\u305b\u3093\u3067\u3057\u305f\u3002<\/p>\n
\"\"<\/a>
\n\u203bn20r20: LeLU 20\u500b\u3001gru 20\u500b, n40r40: LeLU 40\u500b\u3001gru 40\u500b, n40r80: LeLU 40\u500b\u3001gru 80\u500b \u3092\u8868\u3059\u3002<\/small><\/p>\n
\u4ee5\u4e0a\u306e\u3053\u3068\u304b\u3089\u3001\u3053\u306e\u6700\u7d42\u5c64\u3092gru\u306b\u7f6e\u304d\u63db\u3048\u305f\u3060\u3051\u306e\u30e2\u30c7\u30eb\u306f\u3001\u77ed\u671f\u8a18\u61b6\u306f\u3067\u304d\u308b\u3051\u308c\u3069\u3082\u304b\u306a\u308a\u7c21\u5358\u306a\u5834\u5408\u306b\u3057\u304b\u3067\u304d\u306a\u3044\u3068\u3044\u3046\u3053\u3068\u304c\u5206\u304b\u308a\u307e\u3057\u305f\u3002\u3053\u308c\u306flstm \u3067\u3082\u540c\u3058\u3067\u3057\u305f\u3002<\/p>\n
\u30bf\u30b9\u30af\u306b\u306f\u3001L8g34567 \u3068\u3044\u3046L8g67\u3088\u308a\u3082\u3082\u3063\u3068\u96e3\u3057\u3044\u3082\u306e\u3082\u6e96\u5099\u3057\u3066\u3042\u308a\u307e\u3059\u304c\u3001\u5f53\u7136\u3001\u3053\u308c\u3092\u89e3\u304f\u3053\u3068\u306f\u3067\u304d\u307e\u305b\u3093\u3002<\/p>\n
\u3055\u3066\u3001\u6700\u5f8c\u306b\u307e\u3068\u3081\u3067\u3059\u3002<\/p>\n
\u3053\u306e\u8a18\u4e8b\u3067\u306f\u3001\u901a\u5e38\u306eQ\u5b66\u7fd2\u3001\u30cb\u30e5\u30fc\u30e9\u30eb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u3092\u4f7f\u3063\u305fQ\u5b66\u7fd2\u3001\u77ed\u671f\u8a18\u61b6\u4ed8\u304d\u306e\u5f37\u5316\u5b66\u7fd2\u3092\u7d39\u4ecb\u3057\u3001\u305d\u308c\u305e\u308c\u304c\u3067\u304d\u308b\u30bf\u30b9\u30af\u3001\u3067\u304d\u306a\u3044\u30bf\u30b9\u30af\u3092\u660e\u3089\u304b\u306b\u3057\u307e\u3057\u305f\u3002<\/p>\n
\u7279\u306b\u77ed\u671f\u8a18\u61b6\u4ed8\u304d\u5f37\u5316\u5b66\u7fd2gru, lstm \u3067\u306f\u3001\u5f93\u6765\u306e\u5f37\u5316\u5b66\u7fd2\u3067\u304d\u306a\u3044Tmaze_both\u3068Tmaze_either \u304c\u3067\u304d\u308b\u3053\u3068\u3092\u793a\u3057\u307e\u3057\u305f\u3002<\/p>\n
\u305d\u3057\u3066\u3001\u77ed\u671f\u8a18\u61b6\u3092\u4fdd\u6301\u3057\u306a\u304f\u3066\u306f\u3044\u3051\u306a\u3044\u30b9\u30c6\u30c3\u30d7\u6570\u304c\u4f38\u3073\u308b\u3068\u3001\u6025\u6fc0\u306b\u6210\u529f\u78ba\u7387\u304c\u6e1b\u308b\u3053\u3068\u3092\u793a\u3057\u307e\u3057\u305f\u3002L8g34567 \u3084 ruin_2swamp \u306e\u3088\u3046\u306a\u30bf\u30b9\u30af\u3092\u89e3\u304f\u304b\u304c\u4eca\u5f8c\u306e\u8ab2\u984c\u3067\u3059\u3002\u307f\u306a\u3055\u3093\u3082\u305c\u3072\u30c1\u30e3\u30ec\u30f3\u30b8\u3057\u3066\u307f\u3066\u304f\u3060\u3055\u3044\u3002<\/p>\n
\u304a\u308f\u308a<\/p>\n
[top]<\/a> [01]<\/a> [02]<\/a> [03]<\/a> [04]<\/a> [05]<\/a> [06]<\/a> [07]<\/a> [08]<\/a> [09]<\/a> [10]<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
q, qnet, gru, lstm \u306e\u30b3\u30fc\u30c7\u30a3\u30f3\u30b0\u306b\u9593\u9055\u3044\u306f\u306a\u3044\u306e\u3067\u3057\u3087\u3046\u304b\uff1f\u305d\u306e\u78ba\u8a8d\u3092\u3059\u308b\u305f\u3081\u306b\u306f\u3001Q\u5024\u304c\u6b63\u3057\u3044\u5024\u306b\u53ce\u675f\u3059\u308b\u304b\u3092\u30c1\u30a7\u30c3\u30af\u3059\u308b\u3053\u3068\u304c\u5927\u5207\u3067\u3059\u3002 \u307e\u305f\u3001\u77ed\u671f\u8a18\u61b6\u30e6\u30cb\u30c3\u30c8(gru\/lstm)\u306e\u77ed\u671f\u8a18\u61b6\u306e\u80fd\u529b\u306f\u3069\u308c\u304f\u3089\u3044\u306a\u306e\u3067\u3057\u3087\u3046\u304b\u3002\u5b9a\u91cf\u7684\u306b\u77e5\u308a\u305f\u3044\u3067\u3059\u3002 \u3053\u306e\u3088\u3046 […]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":2484,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"categories":[2,44],"class_list":{"0":"post-2983","1":"page","2":"type-page","3":"status-publish","5":"category-python","6":"category-44","7":"cat-2-id","8":"cat-44-id"},"_links":{"self":[{"href":"http:\/\/itoshi.main.jp\/tech\/wp-json\/wp\/v2\/pages\/2983","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/itoshi.main.jp\/tech\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"http:\/\/itoshi.main.jp\/tech\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"http:\/\/itoshi.main.jp\/tech\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/itoshi.main.jp\/tech\/wp-json\/wp\/v2\/comments?post=2983"}],"version-history":[{"count":27,"href":"http:\/\/itoshi.main.jp\/tech\/wp-json\/wp\/v2\/pages\/2983\/revisions"}],"predecessor-version":[{"id":3035,"href":"http:\/\/itoshi.main.jp\/tech\/wp-json\/wp\/v2\/pages\/2983\/revisions\/3035"}],"up":[{"embeddable":true,"href":"http:\/\/itoshi.main.jp\/tech\/wp-json\/wp\/v2\/pages\/2484"}],"wp:attachment":[{"href":"http:\/\/itoshi.main.jp\/tech\/wp-json\/wp\/v2\/media?parent=2983"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/itoshi.main.jp\/tech\/wp-json\/wp\/v2\/categories?post=2983"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}