Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 89 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.5213. Min Range: -1.5213

Max Act: 1.5213. Min Act: -0.1700

Data Index: 5304010 (Open Web Text)

Max Activating Token Index: 400

Text #1

Max Range: 1.5213. Min Range: -1.5213

Max Act: 1.4004. Min Act: -0.1700

Data Index: 6333442 (Open Web Text)

Max Activating Token Index: 409

Text #2

Max Range: 1.5213. Min Range: -1.5213

Max Act: 1.3252. Min Act: -0.1700

Data Index: 5519728 (Open Web Text)

Max Activating Token Index: 370

Text #3

Max Range: 1.5213. Min Range: -1.5213

Max Act: 1.2630. Min Act: -0.1700

Data Index: 3353487 (Open Web Text)

Max Activating Token Index: 387

Text #4

Max Range: 1.5213. Min Range: -1.5213

Max Act: 1.1819. Min Act: -0.1700

Data Index: 5519730 (Open Web Text)

Max Activating Token Index: 386

Text #5

Max Range: 1.5213. Min Range: -1.5213

Max Act: 1.1822. Min Act: -0.1700

Data Index: 7647610 (Open Web Text)

Max Activating Token Index: 401

Text #6

Max Range: 1.5213. Min Range: -1.5213

Max Act: 1.1810. Min Act: -0.1700

Data Index: 8364646 (Open Web Text)

Max Activating Token Index: 336

Text #7

Max Range: 1.5213. Min Range: -1.5213

Max Act: 1.1404. Min Act: -0.1700

Data Index: 7419548 (Open Web Text)

Max Activating Token Index: 376

Text #8

Max Range: 1.5213. Min Range: -1.5213

Max Act: 1.0486. Min Act: -0.1700

Data Index: 3224037 (Open Web Text)

Max Activating Token Index: 395

Text #9

Max Range: 1.5213. Min Range: -1.5213

Max Act: 1.0368. Min Act: -0.1700

Data Index: 8364643 (Open Web Text)

Max Activating Token Index: 367

Text #10

Max Range: 1.5213. Min Range: -1.5213

Max Act: 1.0254. Min Act: -0.1700

Data Index: 6503283 (Open Web Text)

Max Activating Token Index: 408

Text #11

Max Range: 1.5213. Min Range: -1.5213

Max Act: 1.0114. Min Act: -0.1700

Data Index: 2698870 (Open Web Text)

Max Activating Token Index: 394

Text #12

Max Range: 1.5213. Min Range: -1.5213

Max Act: 1.0008. Min Act: -0.1700

Data Index: 1610561 (Open Web Text)

Max Activating Token Index: 372

Text #13

Max Range: 1.5213. Min Range: -1.5213

Max Act: 1.0184. Min Act: -0.1700

Data Index: 7349126 (Open Web Text)

Max Activating Token Index: 374

Text #14

Max Range: 1.5213. Min Range: -1.5213

Max Act: 0.9890. Min Act: -0.1700

Data Index: 570701 (Open Web Text)

Max Activating Token Index: 374

Text #15

Max Range: 1.5213. Min Range: -1.5213

Max Act: 0.9843. Min Act: -0.1700

Data Index: 2163461 (Open Web Text)

Max Activating Token Index: 401

Text #16

Max Range: 1.5213. Min Range: -1.5213

Max Act: 0.9550. Min Act: -0.1700

Data Index: 7487095 (Open Web Text)

Max Activating Token Index: 360

Text #17

Max Range: 1.5213. Min Range: -1.5213

Max Act: 0.9530. Min Act: -0.1700

Data Index: 439938 (Open Web Text)

Max Activating Token Index: 336

Text #18

Max Range: 1.5213. Min Range: -1.5213

Max Act: 0.9203. Min Act: -0.1700

Data Index: 6933373 (Open Web Text)

Max Activating Token Index: 400

Text #19

Max Range: 1.5213. Min Range: -1.5213

Max Act: 0.9225. Min Act: -0.1700

Data Index: 8340599 (Open Web Text)

Max Activating Token Index: 373