Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 795 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.4041. Min Range: -6.4041

Max Act: 6.4041. Min Act: -0.1700

Data Index: 4639039 (Open Web Text)

Max Activating Token Index: 880

Text #1

Max Range: 6.4041. Min Range: -6.4041

Max Act: 6.3164. Min Act: -0.1700

Data Index: 1806358 (Open Web Text)

Max Activating Token Index: 519

Text #2

Max Range: 6.4041. Min Range: -6.4041

Max Act: 6.2930. Min Act: -0.1700

Data Index: 7291328 (Open Web Text)

Max Activating Token Index: 841

Text #3

Max Range: 6.4041. Min Range: -6.4041

Max Act: 6.2203. Min Act: -0.1700

Data Index: 4888746 (Open Web Text)

Max Activating Token Index: 384

Text #4

Max Range: 6.4041. Min Range: -6.4041

Max Act: 6.2479. Min Act: -0.1700

Data Index: 2950935 (Open Web Text)

Max Activating Token Index: 272

Text #5

Max Range: 6.4041. Min Range: -6.4041

Max Act: 6.2006. Min Act: -0.1700

Data Index: 4682146 (Open Web Text)

Max Activating Token Index: 939

Text #6

Max Range: 6.4041. Min Range: -6.4041

Max Act: 6.0833. Min Act: -0.1700

Data Index: 3356146 (Open Web Text)

Max Activating Token Index: 500

Text #7

Max Range: 6.4041. Min Range: -6.4041

Max Act: 6.0683. Min Act: -0.1700

Data Index: 3959833 (Open Web Text)

Max Activating Token Index: 856

Text #8

Max Range: 6.4041. Min Range: -6.4041

Max Act: 6.0607. Min Act: -0.1699

Data Index: 3518906 (Open Web Text)

Max Activating Token Index: 38

Text #9

Max Range: 6.4041. Min Range: -6.4041

Max Act: 5.9910. Min Act: -0.1700

Data Index: 2248926 (Open Web Text)

Max Activating Token Index: 947

Text #10

Max Range: 6.4041. Min Range: -6.4041

Max Act: 5.9258. Min Act: -0.1700

Data Index: 1501496 (Open Web Text)

Max Activating Token Index: 747

Text #11

Max Range: 6.4041. Min Range: -6.4041

Max Act: 5.9686. Min Act: -0.1700

Data Index: 2863000 (Open Web Text)

Max Activating Token Index: 158

Text #12

Max Range: 6.4041. Min Range: -6.4041

Max Act: 5.9483. Min Act: -0.1700

Data Index: 8812065 (Open Web Text)

Max Activating Token Index: 1021

Text #13

Max Range: 6.4041. Min Range: -6.4041

Max Act: 5.9676. Min Act: -0.1700

Data Index: 2779240 (Open Web Text)

Max Activating Token Index: 472

Text #14

Max Range: 6.4041. Min Range: -6.4041

Max Act: 5.8915. Min Act: -0.1700

Data Index: 1428466 (Open Web Text)

Max Activating Token Index: 596

Text #15

Max Range: 6.4041. Min Range: -6.4041

Max Act: 5.9058. Min Act: -0.1700

Data Index: 1930267 (Open Web Text)

Max Activating Token Index: 587

Text #16

Max Range: 6.4041. Min Range: -6.4041

Max Act: 5.9046. Min Act: -0.1700

Data Index: 7563334 (Open Web Text)

Max Activating Token Index: 480

Text #17

Max Range: 6.4041. Min Range: -6.4041

Max Act: 5.8999. Min Act: -0.1700

Data Index: 3624044 (Open Web Text)

Max Activating Token Index: 855

Text #18

Max Range: 6.4041. Min Range: -6.4041

Max Act: 5.9013. Min Act: -0.1700

Data Index: 3417138 (Open Web Text)

Max Activating Token Index: 577

Text #19

Max Range: 6.4041. Min Range: -6.4041

Max Act: 5.8517. Min Act: -0.1699

Data Index: 4104270 (Open Web Text)

Max Activating Token Index: 817