Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2151 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.2329. Min Range: -5.2329

Max Act: 5.2329. Min Act: -0.1700

Data Index: 2014365 (Open Web Text)

Max Activating Token Index: 416

Text #1

Max Range: 5.2329. Min Range: -5.2329

Max Act: 5.1773. Min Act: -0.1700

Data Index: 6552917 (Open Web Text)

Max Activating Token Index: 46

Text #2

Max Range: 5.2329. Min Range: -5.2329

Max Act: 5.1906. Min Act: -0.1700

Data Index: 1473991 (Open Web Text)

Max Activating Token Index: 891

Text #3

Max Range: 5.2329. Min Range: -5.2329

Max Act: 5.1534. Min Act: -0.1700

Data Index: 268959 (Open Web Text)

Max Activating Token Index: 492

Text #4

Max Range: 5.2329. Min Range: -5.2329

Max Act: 5.1344. Min Act: -0.1700

Data Index: 8804401 (Open Web Text)

Max Activating Token Index: 874

Text #5

Max Range: 5.2329. Min Range: -5.2329

Max Act: 5.1444. Min Act: -0.1700

Data Index: 6298996 (Open Web Text)

Max Activating Token Index: 481

Text #6

Max Range: 5.2329. Min Range: -5.2329

Max Act: 5.1317. Min Act: -0.1700

Data Index: 2212084 (Open Web Text)

Max Activating Token Index: 282

Text #7

Max Range: 5.2329. Min Range: -5.2329

Max Act: 5.1507. Min Act: -0.1700

Data Index: 5263236 (Open Web Text)

Max Activating Token Index: 42

Text #8

Max Range: 5.2329. Min Range: -5.2329

Max Act: 5.1177. Min Act: -0.1700

Data Index: 6970231 (Open Web Text)

Max Activating Token Index: 612

Text #9

Max Range: 5.2329. Min Range: -5.2329

Max Act: 5.1073. Min Act: -0.1700

Data Index: 170217 (Open Web Text)

Max Activating Token Index: 677

Text #10

Max Range: 5.2329. Min Range: -5.2329

Max Act: 5.1250. Min Act: -0.1700

Data Index: 1403256 (Open Web Text)

Max Activating Token Index: 182

Text #11

Max Range: 5.2329. Min Range: -5.2329

Max Act: 5.1219. Min Act: -0.1700

Data Index: 7133351 (Open Web Text)

Max Activating Token Index: 80

Text #12

Max Range: 5.2329. Min Range: -5.2329

Max Act: 5.1092. Min Act: -0.1700

Data Index: 7101111 (Open Web Text)

Max Activating Token Index: 60

Text #13

Max Range: 5.2329. Min Range: -5.2329

Max Act: 5.1168. Min Act: -0.1700

Data Index: 2038696 (Open Web Text)

Max Activating Token Index: 832

Text #14

Max Range: 5.2329. Min Range: -5.2329

Max Act: 5.0982. Min Act: -0.1700

Data Index: 4256874 (Open Web Text)

Max Activating Token Index: 423

Text #15

Max Range: 5.2329. Min Range: -5.2329

Max Act: 5.0889. Min Act: -0.1700

Data Index: 238150 (Open Web Text)

Max Activating Token Index: 473

Text #16

Max Range: 5.2329. Min Range: -5.2329

Max Act: 5.1137. Min Act: -0.1700

Data Index: 6026954 (Open Web Text)

Max Activating Token Index: 216

Text #17

Max Range: 5.2329. Min Range: -5.2329

Max Act: 5.1369. Min Act: -0.1700

Data Index: 5100882 (Open Web Text)

Max Activating Token Index: 503

Text #18

Max Range: 5.2329. Min Range: -5.2329

Max Act: 5.1217. Min Act: -0.1700

Data Index: 5396013 (Open Web Text)

Max Activating Token Index: 371

Text #19

Max Range: 5.2329. Min Range: -5.2329

Max Act: 5.0635. Min Act: -0.1700

Data Index: 138234 (Open Web Text)

Max Activating Token Index: 589