Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 855 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.8241. Min Range: -6.8241

Max Act: 6.8241. Min Act: -0.1700

Data Index: 6130943 (Open Web Text)

Max Activating Token Index: 425

Text #1

Max Range: 6.8241. Min Range: -6.8241

Max Act: 6.6959. Min Act: -0.1700

Data Index: 6803093 (Open Web Text)

Max Activating Token Index: 403

Text #2

Max Range: 6.8241. Min Range: -6.8241

Max Act: 6.6348. Min Act: -0.1700

Data Index: 7076532 (Open Web Text)

Max Activating Token Index: 750

Text #3

Max Range: 6.8241. Min Range: -6.8241

Max Act: 6.5438. Min Act: -0.1700

Data Index: 105567 (Open Web Text)

Max Activating Token Index: 1015

Text #4

Max Range: 6.8241. Min Range: -6.8241

Max Act: 6.5548. Min Act: -0.1700

Data Index: 2912706 (Open Web Text)

Max Activating Token Index: 121

Text #5

Max Range: 6.8241. Min Range: -6.8241

Max Act: 6.5217. Min Act: -0.1700

Data Index: 2552859 (Open Web Text)

Max Activating Token Index: 493

Text #6

Max Range: 6.8241. Min Range: -6.8241

Max Act: 6.3521. Min Act: -0.1700

Data Index: 2582576 (Open Web Text)

Max Activating Token Index: 900

Text #7

Max Range: 6.8241. Min Range: -6.8241

Max Act: 6.3620. Min Act: -0.1700

Data Index: 1655991 (Open Web Text)

Max Activating Token Index: 395

Text #8

Max Range: 6.8241. Min Range: -6.8241

Max Act: 6.3329. Min Act: -0.1700

Data Index: 6044700 (Open Web Text)

Max Activating Token Index: 168

Text #9

Max Range: 6.8241. Min Range: -6.8241

Max Act: 6.2052. Min Act: -0.1700

Data Index: 7617998 (Open Web Text)

Max Activating Token Index: 461

Text #10

Max Range: 6.8241. Min Range: -6.8241

Max Act: 6.2195. Min Act: -0.1700

Data Index: 2480715 (Open Web Text)

Max Activating Token Index: 1006

Text #11

Max Range: 6.8241. Min Range: -6.8241

Max Act: 6.2696. Min Act: -0.1700

Data Index: 749260 (Open Web Text)

Max Activating Token Index: 323

Text #12

Max Range: 6.8241. Min Range: -6.8241

Max Act: 6.2561. Min Act: -0.1700

Data Index: 1552795 (Open Web Text)

Max Activating Token Index: 446

Text #13

Max Range: 6.8241. Min Range: -6.8241

Max Act: 6.2318. Min Act: -0.1700

Data Index: 222762 (Open Web Text)

Max Activating Token Index: 151

Text #14

Max Range: 6.8241. Min Range: -6.8241

Max Act: 6.2456. Min Act: -0.1700

Data Index: 743412 (Open Web Text)

Max Activating Token Index: 297

Text #15

Max Range: 6.8241. Min Range: -6.8241

Max Act: 6.2072. Min Act: -0.1700

Data Index: 372372 (Open Web Text)

Max Activating Token Index: 139

Text #16

Max Range: 6.8241. Min Range: -6.8241

Max Act: 6.1460. Min Act: -0.1700

Data Index: 1012883 (Open Web Text)

Max Activating Token Index: 654

Text #17

Max Range: 6.8241. Min Range: -6.8241

Max Act: 6.1830. Min Act: -0.1700

Data Index: 8292277 (Open Web Text)

Max Activating Token Index: 762

Text #18

Max Range: 6.8241. Min Range: -6.8241

Max Act: 6.1429. Min Act: -0.1700

Data Index: 7407295 (Open Web Text)

Max Activating Token Index: 547

Text #19

Max Range: 6.8241. Min Range: -6.8241

Max Act: 6.1176. Min Act: -0.1700

Data Index: 516604 (Open Web Text)

Max Activating Token Index: 612