Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1637 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1557. Min Range: -4.1557

Max Act: 4.1557. Min Act: -0.1700

Data Index: 3311892 (Open Web Text)

Max Activating Token Index: 462

Text #1

Max Range: 4.1557. Min Range: -4.1557

Max Act: 4.0308. Min Act: -0.1700

Data Index: 1587837 (Open Web Text)

Max Activating Token Index: 442

Text #2

Max Range: 4.1557. Min Range: -4.1557

Max Act: 3.9243. Min Act: -0.1700

Data Index: 2624787 (Open Web Text)

Max Activating Token Index: 449

Text #3

Max Range: 4.1557. Min Range: -4.1557

Max Act: 3.9141. Min Act: -0.1700

Data Index: 3671984 (Open Web Text)

Max Activating Token Index: 442

Text #4

Max Range: 4.1557. Min Range: -4.1557

Max Act: 3.9252. Min Act: -0.1700

Data Index: 5640276 (Open Web Text)

Max Activating Token Index: 461

Text #5

Max Range: 4.1557. Min Range: -4.1557

Max Act: 3.8956. Min Act: -0.1700

Data Index: 7462215 (Open Web Text)

Max Activating Token Index: 452

Text #6

Max Range: 4.1557. Min Range: -4.1557

Max Act: 3.8551. Min Act: -0.1700

Data Index: 1081145 (Open Web Text)

Max Activating Token Index: 426

Text #7

Max Range: 4.1557. Min Range: -4.1557

Max Act: 3.8538. Min Act: -0.1700

Data Index: 5645170 (Open Web Text)

Max Activating Token Index: 431

Text #8

Max Range: 4.1557. Min Range: -4.1557

Max Act: 3.8397. Min Act: -0.1700

Data Index: 3985066 (Open Web Text)

Max Activating Token Index: 446

Text #9

Max Range: 4.1557. Min Range: -4.1557

Max Act: 3.8339. Min Act: -0.1700

Data Index: 3815323 (Open Web Text)

Max Activating Token Index: 465

Text #10

Max Range: 4.1557. Min Range: -4.1557

Max Act: 3.8079. Min Act: -0.1700

Data Index: 3446047 (Open Web Text)

Max Activating Token Index: 445

Text #11

Max Range: 4.1557. Min Range: -4.1557

Max Act: 3.8440. Min Act: -0.1700

Data Index: 6052151 (Open Web Text)

Max Activating Token Index: 448

Text #12

Max Range: 4.1557. Min Range: -4.1557

Max Act: 3.8337. Min Act: -0.1700

Data Index: 284953 (Open Web Text)

Max Activating Token Index: 450

Text #13

Max Range: 4.1557. Min Range: -4.1557

Max Act: 3.7720. Min Act: -0.1700

Data Index: 1721776 (Open Web Text)

Max Activating Token Index: 455

Text #14

Max Range: 4.1557. Min Range: -4.1557

Max Act: 3.7336. Min Act: -0.1700

Data Index: 1595630 (Open Web Text)

Max Activating Token Index: 460

Text #15

Max Range: 4.1557. Min Range: -4.1557

Max Act: 3.7508. Min Act: -0.1700

Data Index: 6758943 (Open Web Text)

Max Activating Token Index: 432

Text #16

Max Range: 4.1557. Min Range: -4.1557

Max Act: 3.8005. Min Act: -0.1700

Data Index: 124651 (Open Web Text)

Max Activating Token Index: 470

Text #17

Max Range: 4.1557. Min Range: -4.1557

Max Act: 3.7644. Min Act: -0.1700

Data Index: 1435966 (Open Web Text)

Max Activating Token Index: 403

Text #18

Max Range: 4.1557. Min Range: -4.1557

Max Act: 3.7494. Min Act: -0.1700

Data Index: 776155 (Open Web Text)

Max Activating Token Index: 435

Text #19

Max Range: 4.1557. Min Range: -4.1557

Max Act: 3.7378. Min Act: -0.1700

Data Index: 1081141 (Open Web Text)

Max Activating Token Index: 487