Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 441 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.2767. Min Range: -5.2767

Max Act: 5.2767. Min Act: -0.1700

Data Index: 2501950 (Open Web Text)

Max Activating Token Index: 468

Text #1

Max Range: 5.2767. Min Range: -5.2767

Max Act: 5.2465. Min Act: -0.1700

Data Index: 15335 (Open Web Text)

Max Activating Token Index: 157

Text #2

Max Range: 5.2767. Min Range: -5.2767

Max Act: 5.2628. Min Act: -0.1700

Data Index: 8202933 (Open Web Text)

Max Activating Token Index: 484

Text #3

Max Range: 5.2767. Min Range: -5.2767

Max Act: 5.2150. Min Act: -0.1700

Data Index: 5133757 (Open Web Text)

Max Activating Token Index: 148

Text #4

Max Range: 5.2767. Min Range: -5.2767

Max Act: 5.2192. Min Act: -0.1700

Data Index: 1810202 (Open Web Text)

Max Activating Token Index: 986

Text #5

Max Range: 5.2767. Min Range: -5.2767

Max Act: 5.2393. Min Act: -0.1700

Data Index: 8438922 (Open Web Text)

Max Activating Token Index: 362

Text #6

Max Range: 5.2767. Min Range: -5.2767

Max Act: 5.1841. Min Act: -0.1700

Data Index: 1204756 (Open Web Text)

Max Activating Token Index: 472

Text #7

Max Range: 5.2767. Min Range: -5.2767

Max Act: 5.1811. Min Act: -0.1700

Data Index: 3659473 (Open Web Text)

Max Activating Token Index: 382

Text #8

Max Range: 5.2767. Min Range: -5.2767

Max Act: 5.1778. Min Act: -0.1700

Data Index: 3239537 (Open Web Text)

Max Activating Token Index: 473

Text #9

Max Range: 5.2767. Min Range: -5.2767

Max Act: 5.1898. Min Act: -0.1700

Data Index: 989945 (Open Web Text)

Max Activating Token Index: 199

Text #10

Max Range: 5.2767. Min Range: -5.2767

Max Act: 5.1802. Min Act: -0.1700

Data Index: 1291758 (Open Web Text)

Max Activating Token Index: 552

Text #11

Max Range: 5.2767. Min Range: -5.2767

Max Act: 5.1904. Min Act: -0.1700

Data Index: 8829297 (Open Web Text)

Max Activating Token Index: 361

Text #12

Max Range: 5.2767. Min Range: -5.2767

Max Act: 5.1596. Min Act: -0.1700

Data Index: 7348501 (Open Web Text)

Max Activating Token Index: 335

Text #13

Max Range: 5.2767. Min Range: -5.2767

Max Act: 5.1451. Min Act: -0.1700

Data Index: 8183922 (Open Web Text)

Max Activating Token Index: 203

Text #14

Max Range: 5.2767. Min Range: -5.2767

Max Act: 5.1460. Min Act: -0.1700

Data Index: 3139639 (Open Web Text)

Max Activating Token Index: 196

Text #15

Max Range: 5.2767. Min Range: -5.2767

Max Act: 5.1680. Min Act: -0.1700

Data Index: 1020504 (Open Web Text)

Max Activating Token Index: 91

Text #16

Max Range: 5.2767. Min Range: -5.2767

Max Act: 5.1620. Min Act: -0.1700

Data Index: 1527372 (Open Web Text)

Max Activating Token Index: 615

Text #17

Max Range: 5.2767. Min Range: -5.2767

Max Act: 5.1464. Min Act: -0.1700

Data Index: 6455462 (Open Web Text)

Max Activating Token Index: 150

Text #18

Max Range: 5.2767. Min Range: -5.2767

Max Act: 5.1412. Min Act: -0.1700

Data Index: 6239388 (Open Web Text)

Max Activating Token Index: 517

Text #19

Max Range: 5.2767. Min Range: -5.2767

Max Act: 5.1517. Min Act: -0.1700

Data Index: 8694303 (Open Web Text)

Max Activating Token Index: 750