Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1960 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.3017. Min Range: -6.3017

Max Act: 6.3017. Min Act: -0.1700

Data Index: 8287619 (Open Web Text)

Max Activating Token Index: 78

Text #1

Max Range: 6.3017. Min Range: -6.3017

Max Act: 6.2179. Min Act: -0.1700

Data Index: 1540496 (Open Web Text)

Max Activating Token Index: 832

Text #2

Max Range: 6.3017. Min Range: -6.3017

Max Act: 6.0279. Min Act: -0.1700

Data Index: 7101505 (Open Web Text)

Max Activating Token Index: 730

Text #3

Max Range: 6.3017. Min Range: -6.3017

Max Act: 6.0597. Min Act: -0.1700

Data Index: 1656038 (Open Web Text)

Max Activating Token Index: 223

Text #4

Max Range: 6.3017. Min Range: -6.3017

Max Act: 6.0243. Min Act: -0.1700

Data Index: 3860121 (Open Web Text)

Max Activating Token Index: 954

Text #5

Max Range: 6.3017. Min Range: -6.3017

Max Act: 5.9932. Min Act: -0.1700

Data Index: 1898785 (Open Web Text)

Max Activating Token Index: 237

Text #6

Max Range: 6.3017. Min Range: -6.3017

Max Act: 5.9511. Min Act: -0.1700

Data Index: 8096020 (Open Web Text)

Max Activating Token Index: 333

Text #7

Max Range: 6.3017. Min Range: -6.3017

Max Act: 5.9803. Min Act: -0.1700

Data Index: 2939061 (Open Web Text)

Max Activating Token Index: 819

Text #8

Max Range: 6.3017. Min Range: -6.3017

Max Act: 5.9572. Min Act: -0.1700

Data Index: 537935 (Open Web Text)

Max Activating Token Index: 647

Text #9

Max Range: 6.3017. Min Range: -6.3017

Max Act: 5.8808. Min Act: -0.1700

Data Index: 5303674 (Open Web Text)

Max Activating Token Index: 266

Text #10

Max Range: 6.3017. Min Range: -6.3017

Max Act: 5.9070. Min Act: -0.1700

Data Index: 6701333 (Open Web Text)

Max Activating Token Index: 352

Text #11

Max Range: 6.3017. Min Range: -6.3017

Max Act: 5.8863. Min Act: -0.1700

Data Index: 4517638 (Open Web Text)

Max Activating Token Index: 580

Text #12

Max Range: 6.3017. Min Range: -6.3017

Max Act: 5.9068. Min Act: -0.1700

Data Index: 7552689 (Open Web Text)

Max Activating Token Index: 436

Text #13

Max Range: 6.3017. Min Range: -6.3017

Max Act: 5.7996. Min Act: -0.1700

Data Index: 6983840 (Open Web Text)

Max Activating Token Index: 553

Text #14

Max Range: 6.3017. Min Range: -6.3017

Max Act: 5.8094. Min Act: -0.1700

Data Index: 2311990 (Open Web Text)

Max Activating Token Index: 330

Text #15

Max Range: 6.3017. Min Range: -6.3017

Max Act: 5.8012. Min Act: -0.1700

Data Index: 8227926 (Open Web Text)

Max Activating Token Index: 446

Text #16

Max Range: 6.3017. Min Range: -6.3017

Max Act: 5.8190. Min Act: -0.1700

Data Index: 1329055 (Open Web Text)

Max Activating Token Index: 69

Text #17

Max Range: 6.3017. Min Range: -6.3017

Max Act: 5.8268. Min Act: -0.1700

Data Index: 2885826 (Open Web Text)

Max Activating Token Index: 453

Text #18

Max Range: 6.3017. Min Range: -6.3017

Max Act: 5.8408. Min Act: -0.1700

Data Index: 1081649 (Open Web Text)

Max Activating Token Index: 497

Text #19

Max Range: 6.3017. Min Range: -6.3017

Max Act: 5.8457. Min Act: -0.1700

Data Index: 6146597 (Open Web Text)

Max Activating Token Index: 590